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1.  Introduction 


Manpower  and  Personnel  Integration  (MANPRINT)  type  concerns  have  been  around  for  a  long 
time  in  many  fields,  but  not  by  that  name.  The  term  “MANPRINT”  came  into  being  in  1984. 
Continued  systemic  problems  during  development,  or  following  release  of  the  system  to  soldiers 
in  an  operational  environment,  led  to  an  initial  focus  on  correction  by  increasing  manpower, 
recruiting  more  talented  soldiers,  and  enhancing  training  programs.  It  was  recognized  that  this 
approach,  by  itself,  was  not  leading  to  system  designs  that  maximized  soldier  performance  with 
the  system.  Optimizing  on  system  perfonnance  also  required  focusing  on  human  requirements 
within  the  system. 

1 . 1  Development  of  MANPRINT  Implementation 

In  the  1984-1985  timeframe,  the  search  began  for  people  who  would  implement  MANPRINT 
type  of  evaluations.  Among  those  selected  to  support  this  initiative  after  systems  were 
developed  was  a  cadre  of  research  psychologists  affiliated  with  the  U.S.  Army  Research  Institute 
(ARI).  The  U.S.  Army  had  separate  Test  and  Evaluation  (T&E)  organizations — entities  for 
creation  of  new  systems.  One  was  responsible  for  supporting  developmental  testing;  the  other 
was  responsible  for  supporting  operational  testing.  During  developmental  testing,  problems 
were  identified  and  presumably  corrected.  Our  role  as  a  Field  Element  was  to  support  the  Test 
and  Experimentation  Command  (TEXCOM)  in  its  mission  of  conducting  operational  testing.  As 
individuals  responsible  for  MANPRINT  evaluations,  data  collected  during  operational  testing 
provided  the  U.S.  Army  with  another  look  at  the  system  closer  to  the  time  when  it  would  be 
placed  in  the  hands  of  the  representative  soldiers.  Those  data  served  as  an  additional  opportunity 
to  determine  what  problems  continued  to  exist  as  the  system  began  to  be  used  in  a  more  mission- 
oriented  integrated  form.  In  1985-86,  after  MANPRINT  had  become  formalized,  people  at 
several  of  the  Field  Elements  (including  ours)  were  sent  back  to  the  Washington  area  to  take  a 
3-week  course  on  life-cycle  management  of  system  development.  This  course  covered  the  more 
management-oriented  aspects  of  implementing  MANPRINT.  As  one  tasked  to  implement 
MANPRINT,  much  of  the  information  imparted  seemed  “good  to  know,”  but  not  particularly 
useful  to  our  mission  of  planning  and  collecting  MANPRINT  type  data.  In  those  early  days,  our 
Field  Element  had  about  ten  research  psychologists  and  so  we  were  able  to  divide  ourselves  up  to 
support  TEXCOM  test  directorates.  As  structured  during  this  period,  TEXCOM  was  responsible 
for  planning  and  collecting  test  data  on  systems  under  test  during  the  operational  test  phase;  the 
Operational  Test  and  Evaluation  Command  (OPTEC)  was  responsible  for  perfonning  the 
evaluation  with  that  data.  In  our  role  of  providing  support  for  TEXCOM,  it  was  our  role  to  plan 
and  collect  the  MANPRINT  type  data.  After  the  3  weeks  of  management-oriented  training,  we 
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still  had  to  work  out  the  specific  data  collection  and  analysis  procedures.  In  the  1986-1995 
timeframe,  our  assignments  in  the  MANPRINT  arena  were  managed  through  a  form  of  matrix 
management.  When  a  system  was  under  development  and  some  form  of  operational  testing — 
Limited  User  Test  and  Evaluation  (LUTE),  Initial  Operational  Test  and  Evaluation  (IOTE)  or  a 
Follow-on  Test  and  Evaluation  (FOTE) — was  being  planned,  our  Field  Element  Chief 
approached  us  and  indicated  that  we  were  being  assigned  to  provide  MANPRINT  support  for 
that  test.  While  as  individuals  we  were  targeted  to  support  systems  from  particular  TEXCOM 
directorates,  our  assignments  often  crossed  directorate  lines.  Once  that  test  was  completed,  the 
analyst  was  then  assigned  to  another  system.  This  form  of  support  with  MANPRINT  analysts 
did  create  some  difficulties  as  there  was  always  the  need  to  become  familiar  with  new  systems; 
however,  in  perspective,  it  made  us  continually  think  about  how  we  were  going  to  implement  the 
MANPRINT  data  collection  and  analysis  effort  for  each  specific  emerging  system.  From  that 
point  on  we  were  put  in  contact  with  the  Test  Officer  (TO)  for  the  system  and  he  kept  us 
apprised  of  test  planning  meetings  and  provided  us  with  feedback  to  our  emerging  MANPRINT 
data  collection  plans.  Historically  (see  TEXCOM  memo  73-1,  para  5-3a  and  figure  5-2,  pp.  5-1 
and  5-3)  (7),  the  role  of  TEXCOM  was  to  plan  and  collect  Level  4  data  (see  table  1).  To  support 
TEXCOM,  our  role  was  generally  to  prepare  a  MANPRINT  data  collection  plan,  collect  that  data 
and  summarize  the  data  in  one  or  more  fonns  designated  for  Level  4  data. 

1.2  Purpose 

The  purpose  of  this  report  is  to  provide  an  overview  of  MANPRINT  and  to  explain  with  several 
examples  how  MANPRINT  type  data  is  collected  and  to  identify  some  common  analyses  used 
during  conduct  of  T&E  efforts  for  emerging  U.S.  Anny  systems.  As  will  be  noted  in  this  report, 
MANPRINT  evaluations  of  emerging  systems  includes  identification  of  both  Training  and 
Human  Factors  Engineering  problems.  Each  of  these  areas  of  interest,  in  themselves,  is 
comprised  of  their  own  body  of  theory  and  accompanying  methodological  procedures.  This 
document  is  not  developed  to  provide  the  concepts  or  summarize  the  methodology  they  employ, 
but  rather  to  provide  specific  examples  of  how  information  can  be  collected  within  these  and  the 
other  five  domains  of  interest  discussed  below.  As  such,  the  MANPRINT  analyst  is  an 
applications  generalist  with  backgrounds  largely  in  psychology  and  human  factors  engineering; 
methodology  used  involves  techniques  developed  in  large  part  within  the  social  sciences, 
statistics  and  operations  research.  These  analytic  techniques  have  been  acquired  throughout  the 
course  of  study  in  these  respective  fields  and  are  adapted  to  address  problems  evolving  in 
system-specific  operational  and  maintenance  procedures.  This  report  provides  a  basis  for 
developing  some  familiarity  with  MANPRINT  implementing  procedures.  It  is  not  reasonable  to 
expect  that  this  training  will  make  you  instant  experts  as  MANPRINT  analysts.  The  primary 
way  you  develop  this  expertise  is  to  use  and  adapt  the  techniques  and  examples  presented  in  this 
report  while  conducting  MANPRINT  evaluations  to  systems  on  which  you  have  been  assigned  to 
provide  MANPRINT  support. 
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Table  1.  First  four  levels  of  data. 


Level 

Description 

Possible  Forms 

Examples  of  Content 

Disposition 

Level  1 
data:  “raw 
data” 

Data  in  their 
original  form. 
Results  of  field 
trials  just  as 
recorded. 

Complete  data 
collection  sheets, 
exposed  camera  film, 
voice  recording 
tapes,  original 
instrumentation, 
magnetic  tape  or 
printouts,  original 
videotapes,  filled 
questionnaires, 
interview  notes. 

1 .  All  reported  target 
presentations  and 
detection. 

2.  Clock  times  of  all 

events. 

3.  Azimuth  and  vertical 
angle  from  each  flash 
base  for  each 

flash. 

4.  Recording  tapes  of 
interviews. 

Accumulated 
during  trials 
for  processing. 
Usually  discarded 
after  use.  Not 
ordinarily  given 
to  another 
agency.  Not 
published. 

Level  2 
data: 

“reduced 

data” 

Data  taken  from 
the  raw  form  and 
consolidated. 

Invalid  or 
unnecessary  data 
points  deleted. 

Trials  declared 
“no  test” 

Deleted. 

Confirmed  and 
corrected  data 
collection  sheets, 
film  with  extraneous 
footage  deleted, 
corrected  tapes  of 
printouts,  and 
original  raw  data 
with  “no  test”  events 
marked  out. 

1.  Record  of  all  valid 
detections. 

2.  Start  and  stop  times 
of  all  applicable 
events. 

3.  Computed  impact 
points  of  each  round 
flashed. 

4.  Confirmed  interview 
records. 

Produced  during 
processing. 

Usually  discarded 
after  use.  Not 
published. 

Level  3 
data: 

“ordered 

data” 

Data  which  have 
been  checked  for 
accuracy  and 
arranged  in 
convenient  order 
for  handling. 
Operations  limited 
to  counting  and 
elementary 
arithmetic. 

Spread  sheets, 
tables,  typed  lists, 
ordered  and  labeled 
printouts,  purified 
and  ordered  tape, 
edited  film,  edited 
magnetic  tapes, 
ordered  punch  cards 

1 .  Counts  of  detections 
arranged  in  sets 
showing  conditions 
under  which 
detections  occurred. 

2.  Elapsed  times  by 
type  events. 

3.  Impact  points  of 
rounds  by  condition 
under  which  fired. 

4.  Interview  comments 
categorized  by  type. 

Not  usually 
published  but 
made  available  to 
analysts.  Usually 
stored  in 
institutional 
data  banks.  All 
or  part  may  be 
published  as 
supplements  to 
test  report. 

Level  4 
data: 

“findings” 

or 

“summary 

statistics” 

Data  which  have 
been  summarized 
by  elementary 
mathematical 
operations. 
Operations  limited 
to  descriptive 
summaries;  no 
judgments  or 
inferences.  Does 
not  go  beyond 
what  was 
observed  in  test. 

Tables  or  graphs 
showing  totals,  means, 
medians,  modes, 
maximums, 
minimums,  quartiles, 
deciles,  percentiles, 
curves,  or  standard 
deviations.  Qualitative 
data  in  form  of  lists, 
histographs,  counts  by 
type,  or  summary 
statements. 

1.  Percentage  of 
presentations 
detected. 

2.  Mean  elapsed  times. 

3.  Calculated  probable 
errors  about  the 
centers  of  impact  or 
conditions. 

4.  Bar  graph  showing 
relative  frequency 
of  each  category  of 
comment. 

Published  as  the 
basic  factual 
findings  of  test 
report. 

3 


2.  Method 


Following  the  beginning  of  the  U.S.  Army’s  Integrated  Test  and  Evaluation  (ITE)  initiative 
around  1996,  the  author  was  assigned  to  provide  MANPRINT  support  to  one  of  the  pilot 
systems — the  All-Source  Analysis  System  (ASAS)  Block  2  (Modifications).  With  this 
assignment,  greater  continuity  came  in  the  system  evaluation  process  as  support  shifted  to 
different  sub-systems  of  ASAS.  From  that  point  until  the  Block  2  IOTE  in  2005,  with  one 
exception,  MANPRINT  support  was  for  ASAS  sub-systems. 

In  1997,  the  Intelligence  Directorate  at  Fort  Huachuca  contacted  the  Fort  Hood  Field  Element 
and  requested  guidance  on  how  to  implement  MANPRINT.  With  the  sizable  number  of 
MANPRINT  support  efforts  that  had  been  conducted  up  through  1996,  the  immediate  question 
asked  was  why  those  evaluation  efforts  could  not  be  used  as  the  basis  for  the  guidance  they 
sought.  The  answer  derived  largely  from  the  distinction  between  reports  that  deal  with  the 
process — the  specific  ways  the  data  are  obtained — and  those  that  focus  on  product — the 
identification  of  specific  MANPRINT  problems  that  follow  from  an  assessment  of  findings  from 
a  body  of  reports  that  had  addressed  MANPRINT  problems.  As  noted  above,  our  early 
affiliation  with  TEXCOM  led  to  more  process-oriented  expertise.  For  the  most  part,  existing 
System  Evaluation  Reports  (SER)  detailed  methodology  about  how  those  results  were 
obtained — the  process — was  often  lacking  or  presented  with  a  “light  brush.”  Study  of  the 
findings  presented  in  the  more  product-assessment-oriented  reports  can  lead  to  identification  of 
information  types  that  need  to  be  collected,  but  generally  do  not  provide  a  clear  picture  of  the 
instruments’  structure  or  how  they  are  used  to  collect  that  information.  While  procedures 
outlined  in  this  tutorial  are  based  on  a  sizable  number  of  emerging  systems  that  were  under  test, 
in  the  author’s  experience,  there  did  not  seem  to  be  a  single  set  of  procedures  for  providing 
MANPRINT  support.  In  some  cases,  the  specific  examples  presented  are  system-specific  (e.g., 
specific  system  operational  and  maintenance  tasks)  and  are  designed  more  to  give  the  evolving 
MANPRINT  analyst  a  feel  for  the  type  and  level  of  detail  reflected  in  the  systems  under  study;  in 
other  cases,  the  infonnation  collected  is  generic  and  has  direct  applicability  to  multiple  systems. 

It  should  be  understood  that  the  examples  provided  are  just  that — examples — and  judgment  must 
always  be  exercised. 

This  section  provides  an  overview  (or  characterization)  of  MANPRINT.  Also  presented  are 
MANPRINT  models  for  evaluation  and  some  recommended  forms  that  can  be  used  as  beginning 
examples  with  each  of  those  process-oriented  models.  Appendices  provide  some  alternate  data 
collection  forms,  cite  alternative  domain-specific  detailed  data  collection  areas,  presents  some  of 
the  more  administrative  procedures  directly  supporting  the  data  collection  implementation  and 
provides  a  sample  analysis  that  has  been  particularly  useful  in  addressing  soldier  sample¬ 
representativeness.  While  the  MANPRINT  Analyst  is  involved  in  the  data  collection  effort,  it 


4 


should  become  clear  that  this  person  will  not  generally  provide  the  same  level  of  support  for  each 
domain  (see  AR  602-2,  table  3-1,  p  1 1)  (2).  That  table  indicates  that  the  major  proponency  for 
data  collection  in  each  of  the  domains  is  assigned  according  to  the  Acquisition  Category  (ACAT) 
and  whether  the  system  under  test  is  an  Integrated  Concept  Team  (ICT)  or  an  Integrated  Product 
Team  (IPT)  test.  For  the  most  part,  the  primary  responsibility  of  the  MANPRINT  analyst  to 
whom  this  tutorial  is  directed  is  in  the  Human  Factors  Engineering  domain.  For  the  other 
domains,  his  responsibility  is  more  that  of  a  data  collector  and  observer  who  reports  problems 
that  appear  to  be  attributed  to  a  domain.  In  those  cases,  the  major  proponent's  responsibility  is  to 
review  and  validate  this  data  and  include  it  in  his  assessment  with  his  analyses  In  those  cases, 
the  pieces  of  information  reported  by  the  MANPRINT  Analyst  are  combined  with  more  detailed 
data  collected  and,  according  to  the  “intersect  theory  of  assessment”  (3),  serve  as  confirmation 
(or  contradiction)  to  the  proponent’s  data  collection  and  analysis  efforts. 

This  tutorial  was  originally  designed  to  be  presented  to  groups  of  individuals  who  desired 
“heads-up”  training  as  MANPRINT  analysts;  however,  in  this  presentation,  the  material  has  been 
restructured  to  be  better  suited  for  individual  review  and  orientation.  This  restructuring  was 
completed  with  the  recognition  that  those  who  might  serve  as  facilitators  for  training  several 
individuals  probably  have  a  level  of  expertise  in  MANPRINT  evaluation  that  they  would  prefer 
to  use  and  in  a  manner  more  suited  to  their  experiences.  Like  the  proverbial  “blind  men  trying  to 
describe  an  elephant,”  different  people  will  have  differing  opinions. 

2.1  Characterization  of  MANPRINT 

MANPRINT  has  been  defined  as  Manpower  and  Personnel  Integration.  At  present,  MANPRINT 
is  characterized  by  seven  domains,  or  areas  of  interest.  Discussion  in  this  section  provides  a 
more  general  description  of  matters  and  issues  addressed  in  each  of  these  areas  of  interest. 

2.1.1  Manpower  Domain 

Within  this  area  of  interest,  the  basic  question  addressed  is:  Are  there  enough  soldiers  to  operate 
and  maintain  the  system?  Factors  addressed  when  considering  this  area  are  the  workload  and 
types  of  job  stress  associated  with  perfonning  system  specific  critical  tasks.  Other  sources  of 
problems  that  may  be  related  to  the  Manpower  Domain  are  based  on  soldiers’ judgments  about 
the  probable  causes  for  reported  problems. 

The  definition  of  manpower  in  terms  of  the  Training  and  Doctrine  Command  (TRADOC) 
Systems  Manager  (TSM)  and  Combat  Developer  (CD)  is  to  provide  the  guidelines  of  how  many 
people  the  system  needs  to  be  operated  and  maintained.  MANPRINT  analysts  don’t  actually  set 
any  requirements.  The  MANPRINT  analyst  looks  at  what  the  operators  and  maintainers’ 
performance  evidences  (usually  through  reported  problems  and  perfonnance  deficiencies)  and 
then  makes  a  judgment  about  whether  the  U.S.  Army  should  rethink  how  many  soldiers  they 
need  to  do  the  job — operate  and  maintain  the  system.  That  information  is  provided  to  the  U.S. 
Army  Test  and  Evaluation  Command  (ATEC)  with  supporting  rationale. 
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2.1.2  Personnel  Domain 

Within  this  area  of  interest,  there  are  three  major  questions  addressed:  (1)  Do  we  have  the  right 
type  of  soldiers  “manning”  the  system  (MOS/skill  level)?;  (2)  Are  soldiers  participating  in 
developmental  tests  among  the  best  of  those  who  will  operate  and  maintain  the  system? 
Comments  of  those  soldiers  are  needed  to  identify  problems  with  the  system  that  may  be 
especially  troublesome  for  the  aver  age  soldier  and  to  motivate  systemic  changes  in  procedures 
and/or  equipment  before  it  is  placed  in  the  hands  of  the  representative  soldier;  and  (3)  Are  the 
test  players  in  the  Operational  Test  representative  of  the  target  audience  for  whom  the  system  is 
intended?  The  MOS  selected  for  the  system  under  study  are  generally  those  who  had  been 
operating  and  maintaining  the  predecessor  system.  This  domain  addresses  whether  those 
soldiers  have  the  appropriate  abilities  and  the  right  skills.  When  soldiers  report  a  particular 
problem  may  have  occurred  because  the  “right”  type  of  soldiers  are  not  “manning”  the  system, 
this  suggests  that  the  source  of  the  problem  may  be  attributed  to  the  Personnel  Domain  ( 4 ).  In 
developmental  testing,  it  is  best  to  use  “golden  crews.”  In  earlier  tests  and  evaluations,  it  had 
been  noted  that  these  superior  crews  were  sometimes  used  in  operational  testing  and  had  the 
consequences  of  making  the  system  appear  better  than  it  turned  out  to  be  when  it  was  placed  in 
the  hands  of  the  more  representative  soldier.  Those  are  the  people  you  want  in  developmental 
testing  because  they  will  identify  the  problems  that  exist  in  the  system  and  are  likely  to  cause 
problems  for  the  typical  soldiers  who  will  operate  and  maintain  the  system.  In  an  operational 
test,  you  want  representative  (“typical”)  soldiers.  To  make  that  kind  of  judgment,  procedures  for 
determining  what  representative  means  must  be  developed.  Addressing  the  latter  two  major 
questions  involves  exploring  the  demographic  background  characteristics  and  ability  measures 
like  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  scores  of  the  selected  soldiers  and 
comparing  them  with  those  exhibited  by  the  ones  who  operated  and  maintained  the  predecessor 
system. 

Major  resources  used  to  make  comparisons  between  the  test  sample  selected  and  the  population 
(target  audience)  for  whom  the  intended  system  is  being  designed  involves  contacting  a 
Department  of  Defense  repository  of  this  information.  Previous  efforts  have  indicated  that  the 
Defense  Manpower  Data  Center  (DMDC)  in  Monterey,  CA,  is  a  responsive  source.  Following 
September  1 1 ,  DMDC  reorganized  and  new  emerging  procedures  have  been  established  for 
obtaining  demographic  data  and  ASVAB  scores  for  active  duty  enlisted  soldiers  and 
demographic  data  for  active  duty  officers. 
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Details  for  contacting  DMDC  include  the  following  information: 

•  Initial  POC:  Michelle  Rudolph,  Branch  Chief  for  Personnel  and  Manpower 

•  Phone:  (831)  583-2400;  DSN  878-2951;  FAX  (831)  583-2340 

•  Address:  DMDC,  DOD  Center,  Monterey  Bay,  Seaside,  CA  93955-6771 

•  Establish  Request:  Data  Request  System  (DRS):  https://www.dmdc.osd.mil/drs/ 

•  Enter:  DMDC  Assigned  Name  and  Password 

•  Enter:  Request  Title 

•  Choose  Subject:  e.g.,  Accessions 

•  Check  Groups:  e.g.,  Active  Duty  Military  and  MEPCOM 

•  Enter  Detailed  Description:  “Support  MANPRINT  Assessment  in  Personnel  Domain 

for  the _ System” 

-Reference  memorandum  documenting  request  (sample  presented  in  appendix  A). 

-When  contacted  by  DMDC  analyst,  reference  that  analyst  in  the  memo  ATTN  line. 

Appendix  B  provides  a  sample  of  how  information  provided  by  DMDC  can  be  used  to  address 
the  sample  representativeness  of  the  soldiers  participating  in  an  operational  test. 

2.1.3  Training  Domain 

Within  this  area  of  interest,  the  major  question  addressed  is  the  adequacy  of  the  training  provided 
to  soldiers  to  operate  and  maintain  the  system.  The  major  parameters  addressed  in  a  training 
evaluation  include: 

1 .  What  is  being  trained 

a.  Individual  tasks 

b.  Collective  tasks 

2.  What  is  the  type  of  training  evaluation 
a.  Process 

i.  Training  environment  adequacy 

ii.  Instructional/leaming  difficulties 
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b.  Product 


i.  Performance  (critical  task  time  and  error) 

ii.  Soldier  problem  reports 

3.  When  does  the  training  evaluation  occur 

a.  Training  Classroom 

b.  Collective  Training  environment  (prior  to  Record  Test) 

c.  Mission  Enclave  (at  end  of  Record  Test) 

The  evaluation  focuses  on  both  individual  tasks— more  generally  the  procedures  for  manipulation 
of  system  software — and  collective  tasks — more  generally  the  mission-oriented  tasks  that  are 
affected  by  performing  sequences  of  individual  tasks  in  specific  orders.  In  teaching  each  of  these 
types  of  tasks,  process  and  product  evaluations  are  conceptually  possible.  In  the  process 
evaluation,  there  is  generally  an  examination  of  the  adequacy  of  the  training  environment  and 
review  of  instructional  and  learning  difficulties.  In  the  product  evaluation,  tasks’  perfonnance 
(time  and  error  scores)  are  reviewed  to  permit  a  judgment  about  deficiencies  in  the  method  of 
instruction  or  specific  reasons  soldiers  experience  learning  difficulties.  In  the  training 
environments  (classroom  and  mission  facilities),  product  evaluation  may  include  hands-on 
exercises  and/or  performance-based  written  tests.  During  the  test  where  soldiers  are  performing 
in  a  mission-oriented  environment,  they  are  asked  to  make  judgments  about  their  performance 
and  indicate  causes  for  perfonnance  difficulties.  Those  causes  can  be  attributed  to  one  or  more 
domains,  and  problems  experienced  in  one  domain  may  be  caused  by  problems  in  another 
domain. 

2,1.4  Human  Factors  Engineering  Domain  (5-7) 

Within  this  area  of  interest,  three  basic  questions  about  the  operational  and  maintenance 
procedures  asked  of  the  emerging  system  include:  (1)  Is  the  equipment/hardware  difficult  or 
complicated  to  use  by  the  soldier?;  (2)  Are  there  problems  with  the  software  used  in  the  system?; 
and  (3)  Are  there  problems  with  the  procedures? 

Evaluation  within  this  domain  focuses  on  hardware/software  interface  problems  used  to 
complete  system-required  individual  and  collective  tasks.  Method  of  evaluation  within  this 
domain  generally  varies  from  mapping  soldiers'  description  of  problems  with  different  critical 
tasks,  presented  as  apparent  causes  for  problems,  to  use  of  a  whole  host  of  different  surveys. 
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Those  surveys  may  focus  on  equipment  components  and  their  physical  characteristics,  such  as 
those  included  in  appendices  C  (5)  and  D  (9).  They  may  also  focus  on  menus  and  pull-down 
menus,  procedures,  function-specific  software  actions,  or  specific  soldier  physical  dimensions. 
In  those  surveys,  judgments  about  these  areas  of  focus  may  ask  about  their  level  of  adequacy  or 
effectiveness.  Where  judgments  are  negative,  soldiers  are  encouraged  to  describe  the  reasons  on 
the  survey  form  or  in  scheduled  follow-up  interviews.  Many  of  the  systems  used  today  have 
computer  equipment  and  software  to  complete  MOS-based  tasks.  In  some  sense,  completing 
those  tasks  as  they  were  originally  trained— in  a  “manual”  (or  analogue)  mode  (without  a 
computer)  involved  different  sets  of  activities  (procedures)  that  often  do  not  parallel  those  that 
are  required  when  those  tasks  are  completed  with  a  series  of  computer  manipulations  using  the 
software.  This  lack  of  parallelism  can  create  problems  in  both  learning  and  executing  the 
intended  tasks  so  that  they  are  consistent  with  how  the  soldier  conceptually  thinks  about 
completing  the  tasks  to  perform  mission  objectives.  Many  times,  the  procedures  for  completing 
the  mission  without  automation  are  quite  different  from  those  used  when  there  is  computer 
augmentation. 

2.1.5  Health  Hazards  Domain  (10,  IT) 

Within  this  area  of  interest,  the  major  question  asked  is:  Does  operation  or  maintenance  of  the 
system  effect  soldier  safety?  For  both  this  domain  and  the  next,  soldiers'  reports  of  problems 
operating  and  maintaining  the  equipment  and  Test  Incident  Reports  (TIRs)  serve  as  the  two 
primary  sources  of  data  collected  by  the  MANPRINT  Analyst  that  tend  to  serve  as  confirmation 
of  known  problems  or  initial  identification  of  problems  needing  further  exploration. 

2.1.6  System  Safety  Domain  (10,  IT) 

Within  this  area  of  interest,  the  major  question  asked  is:  Does  operation  or  maintenance  of  the 
system  affect  safety  of  the  system? 

2.1.7  Soldier  Survivability  Domain 

Within  this  area  of  interest,  the  MANPRINT  analyst  focuses  on  soldiers’  operational  and 
maintenance  problems  while  wearing  Mission  Oriented  Protective  Posture  (MOPP)  gear.  Four 
additional  areas  of  interest  within  this  domain  include:  (1)  Nuclear  Survivability  Testing;  (2) 
Biological  and  Chemical  Survivability  Testing;  (3)  Information  Warfare  Testing;  and  (4) 
Electronic  Warfare  Testing.  As  the  MANPRINT  analyst  generally  does  not  have  either  the 
specific  skills  or  equipment  to  address  these  latter  areas  of  interest,  their  assessment  usually  is 
coordinated  with  the  U.S.  Army  Research  Laboratory  (ARL)  Survivability/Lethality  Analysis 
Directorate  (SLAD)  and  ATEC. 
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2.2  MANPRINT  Models  for  Evaluation 


Based  on  review  of  the  MANPRINT  evaluation  data  collected  primarily  by  the  author  for  several 
systems  (mostly  in  the  1986-1995  period),  it  seemed  that  the  procedures  used  could  generally  be 
characterized  as  one  of  four  types.  Table  2  shows  these  four  types  (models)  as  a  consequence  of 
two  factors:  (1)  Availability  of  System  Critical  Tasks  and  (2)  Level  of  Verbal  Skills  of  those  for 
whom  the  system  was  designed.  Superscripts  in  table  3  are  footnote  references  for  sample 
systems  shown  in  table  2  that  identify  general  planning  factors  for  conducting  the  MANPRINT 
evaluation;  for  the  Heavy  Equipment  Transporter  System  (HETS)  IOTE  the  superscript  also 
indicates  the  multiplier  effect  of  providing  additional  support  to  the  MANPRINT  data  collection 
effort — “what  you  can  do  with  the  resources  you’ve  got.”  In  most  cases,  entries  in  this  matrix 
are  the  names  of  the  systems  under  test  for  which  the  indicated  MANPRINT  Evaluation  model 
was  used  by  the  author.  While  these  factors  are  helpful  to  provide  initial  direction  on  which 
MANPRINT  model  to  use  in  an  evaluation  effort,  it  is  important  to  remember  that  “every  test  is 
different.” 


Table  2.  Models  for  MANPRINT  evaluation  (but  ‘‘every  test  is  different”). 


High  Verbal  Skills 

Low  Verbal  Skills 

Critical  Tasks 
Available 

Model  1 

Various  ASAS  LUTE  and  Block  2 
IOTE 11  (16) 

Model  2  HETS  IOTEb  (13) 
Enhanced  Position  Location  Reporting 
System  (EPLRS)  IOTEc  (1 7) 

Critical  Tasks 

NOT  Available 

Model  3 

War  Fighting  Rapid  Acquisition 
Program  (WRAP):  ASASd  (12) 

Model  4  MlA2e  (14) 

Secure  Mobile  Anti- Jam  Reliable 
Tactical  Terminal  (SMART-T)f  (15) 

Note:  Footnotes  a-f  are  explained  in  table  3. 


Citations  for  the  systems  listed  in  table  2  are  presented  in  the  references  section  (12-17).  Data 
collection  procedures  employed  for  the  MANPRINT  Evaluations  with  each  model  follow  from 
consideration  of  the  characteristics  defining  the  model  and  the  resources/conditions  existing  for 
the  system  being  evaluated.  The  body  of  this  report  explains  the  more  salient  features  of  the 
different  data  collection  procedures  for  each  model.  In  addition  to  this  discussion,  several 
appendices  are  used  to  present  alternative  forms — some  requesting  infonnation  across 
MANPRINT  Domains,  others  requesting  more  detailed  type  of  infonnation  from  specific 
domains.  While  MANPRINT  Analysis  Results  are  usually  presented  by  the  MANPRINT 
Domain,  events  leading  up  to  the  Record  Test  have  a  logical  sequence  and  tend  to  direct  the 
order  in  which  data  collection  proceeds.  As  the  purpose  of  this  paper  is  to  guide  the  planning  for 
implementing  MANPRINT  data  collection,  the  focus  of  this  tutorial  is  on  sample  forms  used 
within  each  model  in  the  order  that  they  are  used. 
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Table  3.  Footnotes  referenced  in  table  2. 


Footnote 

Model 

Who 

Test 

Respondents 

Test 

Parameters 

MANPRINT  Support 
Team 

MANPRINT  Functions 
Provided 

a 

1 

OHH 

Various  ASAS 
LUTE  and 

Block  2  IOTE 

40  (half  being 
data  collectors) 

5-6  days 

1  GS12  (MGR) 

Develop  DC  plan 

Design  DC  instruments 

Collect  and  process  data 

Provide  analyses  specs 

Write  report  (input  to  TER) 

b 

2 

OHH 

HETS-IOTE 

50  (half  being 
data  collectors) 

6  months 

1  GS12  (MGR) 

1  contractor  w/sys 
knowledge  and  data 
processing  skills 

1  Temp  w/yrs  of  test 
experience 

Develop  DC  plan 

Design  DC  Instruments 

Collect  and  Process  Data 

Data  Analysis 

Write  Report  (input  to  TR) 

c 

2 

OHH 

EPLRS-IOTE 

200  soldiers, 
variety  of  MOSs 
and  skill  levels 

1  month 

3  phases 

1  GS12  (MGR) 

3  GS5  temps 

1  experienced 

4  military 

Develop  DC  Plan 

Design  DC  instruments 
collect  and  process  data 

Provide  OEC  validated 

Level  3  Database 

d 

3 

OHH 

WRAP:  ASAS 

10  soldiers 

3  days 

1  GS12 

Develop  DC  plan 

Design  DC  instruments 

Collect  and  process  data 

Report  findings 

e 

4 

NDS 

M1A2-IOTE 

80  armor  MOS 

4  months 

2  phases 

1  GS13  (MGR) 

1  Temp  (CHF) 

Senior  retired  NCO  w/sys 
experience,  4  temps 

Ret  senior  NCOs 
Background  in  system 
TEXCOM  did  data  entry 

Develop  DC  plan 

Design  DC  instruments 

Collect  and  process  data 

Provide  OEC  Validated 

Level  3  database 

f 

4 

OHH 

SMART-T 

30  soldiers 

3  days 

1  GS12 

Develop  DC  plan 

Design  DC  instruments 

Collect  and  process  data 
report  findings 

2.3  Resources  and  Data  Collection  Procedures  for  Conducting  MANPRINT  Evaluations 
With  Four  Different  Models 

The  MANPRINT  Evaluation  models  have  been  presented  in  table  2.  Entries  within  this  table 
are,  with  one  exception,  systems  for  which  the  author  was  requested  to  provide  MANPRINT 
support.  Models  2  and  4  were  used  to  support  MANPRINT  evaluations  for  two  systems  each. 

In  each  case,  systems  supported  by  these  evaluation  models  have  similarities,  but  differ  in  the 
resources  available  and  (for  model  2)  the  conditions  of  testing  (see  table  3). 

2.3.1  Use  of  Model  1  for  MANPRINT  Evaluation  of  the  All-Source  Analysis  System 
(ASAS) 

The  model  designated  as  model  1  appears  to  work  well  with  soldiers  who  have  high  verbal 
skills — like  those  who  operate  and  maintain  the  All  Source  and  Analysis  System  (ASAS) — when 
individual  and  collective  tasks  have  been  identified.  When  approaching  a  new  system  to  conduct 
a  MANPRINT  evaluation,  it  is  important  to  make  a  judgment  about  the  verbal  skill  of  the 
soldiers  with  whom  you  will  be  dealing  and  whether  or  not  system  trainers  can  provide  you  with 
a  list  of  critical  tasks  used  by  the  system. 

2.3. 1 . 1  Resources  Used  for  MANPRINT  Evaluation  of  ASAS.  There  were  about  40  system 
participants — half  of  whom  were  data  collectors — for  a  5-6-day  test.  Except  for  the  New 
Equipment  Test  Team  (NETT)  instructors,  all  respondents  were  soldiers  assigned  a  relevant 
Military  Intelligence  (MI)  Military  Occupational  Specialty  (MOS).  The  MANPRINT  support 
team — the  resources  available  to  conduct  the  MANPRINT  evaluation — involved  just  one 
MANPRINT  analyst.  What  was  one  MANPRINT  analyst  able  to  do?  He  drew  up  the  data 
collection  plans,  designed  the  data  collection  instruments,  collected  and  processed  the  data, 
analyzed  the  data  and  wrote  draft  MANPRINT  input  for  the  System  Analysis  Report  (SAR)  and 
System  Evaluation  Report  (SER). 

2.3. 1 .2  Data  Collection  Procedures  for  MANPRINT  Evaluation  of  ASAS.  In  implementing  the 
MANPRINT  evaluation  using  Model  1 ,  it  was  frequently  helpful  to  develop  an  Event  Design 
Plan  (EDP)  to  guide  the  progression  of  the  evaluation  events.  An  example  of  this  plan  is  shown 
in  table  4.  That  table  was  developed  specifically  to  support  an  assessment  for  an  ASAS  test. 
Comparable  tables  were  developed  for  each  of  the  ASAS  tests  supported  (LUTEs  and  the  Block 
2  IOTE).  It  provided  a  simple  way  to  guide  what  data  is  needed,  when  to  collect  it  and  from 
whom  to  collect  it.  Samples  of  the  fonns  used  are  presented  in  the  same  sequence  that  they  are 
presented  in  this  EDP.  The  first  column  in  this  EDP  is  the  Fonn  Name.  File  names  identify  the 
particular  instrument.  In  this  test,  there  were  several  functionalities;  those  are  listed  in  the 
second  column.  The  third  column  refers  to  the  time  when  you  use  each  form.  The  fourth 
column  indicates  who  are  to  provide  responses  to  items  on  this  form,  and  the  last  column  is  the 
purpose  of  using  the  instruments.  As  already  noted  the  purpose  of  this  paper  is  to  direct  the 
planning  for  implementing  MANPRINT  data  collection,  the  focus  of  this  presentation  is  on 
sample  forms  used  within  each  model  in  the  order  that  they  are  used. 
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Table  4.  Event  design  plan  (EDP)  for  the  all-source  analysis  system  (ASAS)  block  2  Remote  Work  Station  (RWS)  limited  user  test  and  evaluation  (LUTE)  data 
collection  plan. 


Form  Name 

Critical  Task  List 

Time  Used 

Respondents 

Purpose 

Training  Evaluation  Debrief 
(TADB  RWS.WP6) 
(TADB  AMR.WP6) 
(TADB  EAR.WP6) 
(TADBSAR.WP6) 

Remote  Work  Station-RWS 
Asset  Management 

EL1NT  Analysis 

System  Administration 

At  end  of  each  week  for  each 
instructional  block  for  each  set  of 
students 

Analysts 

Data  Collectors 
Instructors 

Identify  tasks  on  which 
training  problems  exist 

ASAS  Training  Process 
Questionnaire 

(ASTNGQ.WP6) 

All  Critical  Tasks 

At  end  of  each  instructional  block 
for  each  set  of  students 

Used  with  (after)  completing 
“Training  Assessment  Debrief’ 

Analysts 

Data  Collectors 

Document  problems 
identified  during  training 

Demographic  Questionnaire 
(PLY  SDEMR.  WP6) 

— 

At  beginning  of  training 

Analysts 

Data  Collectors 

Document  who  are  analysts 
and  data  collectors 

MANPRINT  Debrief 
(DB  RWS.WP6) 

(DB  AMR.WP6) 

(DB  EAR.WP6) 
(DBSAR.WP6) 

Remote  Work  Station-RWS 
Asset  Management 

EL1NT  Analysis 

System  Administration 

At  end  of  each  test  day 
(Pilot  &  Record  Test) 

Analysts 

Data  Collectors 

Document  tasks  performed 
and  those  which  were 
problematic 

MANPRINT  Evaluation 
(MPASSMR.WP6) 

— 

Used  in  conjunction  with  (after) 
“MANPRINT  Debrief’.  One  form 
completed  for  each  problem 
report— at  end  of  each  test  day 
(Pilot  &  Record  Test) 

Analysts 

Data  Collectors 

Provide  detailed  ratings  and 
comments  on  problematic 
tasks  for  the  six  original 
domains  of  MANPRINT 

Table  4.  Event  design  plan  (EDP)  for  the  all-source  analysis  system  (ASAS)  block  2  Remote  Work  Station  (RWS)  limited  user  test  and  evaluation  (LUTE)  data 
collection  plan  (continued). 


Form  Name 

Critical  Task  List 

Time  Used 

Respondents 

Purpose 

ASAS  Human  Factors 
Engineering  (General 
Software  Functioning) 
questionnaire  (OEC 
provided— 61  items) 
(SSHFEQ.WP6) 

— 

Final  Debrief— day  after  last  Record 
Test  day 

Analysts 

Data  Collectors 

Assess  human  factors 
problems 

ASAS  Human  Factors 
Engineering  (Function- 
Specific)  Questionnaire 
for  each  ASAS  Function 

RWSHFEFunctionsSpecificquest 

ions.doc 

AssetManagementHFEFunctions 

Specificquestions.doc 

ELINTHFEFunctionsSpecificque 

stions.doc 

COMINTHFEFunctionsSpecific 

questions.doc 

SystemAdministrationHFEFuncti 

onsSpecificquestions.doc 

Final  Debrief— day  after  last  Record 
Test  day 

Analysts 

Data  Collectors 

Final  problem  report  on 
system  use 

ASAS  Interoperability 
Problems 

interop3-ace_iote_blk2.doc 

Final  Debrief— day  after  last  Record 
Test  day 

Analysts 

Data  Collectors 

Document  problems 
receiving/sending  different 
message  types  (intra/inter 
BFA) 

Final  Debrief  ASAS 

Single  Source  Operator 
/Analyst  Questionnaire 
(SSOAR.WP6) 

— 

Final  Debrief— day  after  last  Record 
Test  day 

Analysts 

Final  problem  report  on 
system  use 

Final  Debrief  ASAS 

Single  Source  Data 
Collector  Questionnaire 
(SSDCR.WP6) 

— 

Final  Debrief— day  after  last  Record 
Test  Day 

Data  Collectors 

Final  problem  report  on 
system  use 

2. 3. 1.2.1  Training  Evaluation  Debrief  (table  5).  The  instruments  in  the  first  row,  first  column  of 
table  4  are  for  part  of  the  training  process  evaluation  in  the  four  ASAS  functionalities  being 
addressed.  The  sample  of  this  form  references  tasks  for  the  Remote  Work  Station  (RWS) 
Functionality.  They  are  recommended  for  use  at  the  end  of  each  week  for  an  instructional  block 
of  training  for  each  set  of  students.  The  MANPRINT  Analyst  needs  to  coordinate  with  the 
NETT  leader  to  determine,  when  exactly,  would  be  the  best  time  to  come  in  and  survey  these 
soldiers  and  to  indicate  those  tasks  that  had  been  trained  during  that  week.  Conducting  surveys, 
at  least  on  a  weekly  basis,  is  desirable  so  that  soldiers  who  have  a  problem  can  provide  relatively 
fresh  knowledge  about  it;  otherwise  new  problems  tend  to  merge  with  the  old  and  then  tend  to  be 
forgotten. 

2. 3. 1.2. 2  Training  Process  Questionnaire.  The  second  row,  first  column  of  table  4  lists  an 
evaluation  instrument  designed  to  address  the  training  process.  This  form  specifically  addresses 
different  aspects  of  the  training  environment.  It  has  been  generally  an  instrument  used  at  the  end 
of  each  instructional  block  for  NETT  (individual  tasks)  type  of  training — in  contrast  to  weekly  as 
with  the  training  evaluation  debriefs.  Potentially,  analysts,  data  collectors  and  instructors  can  all 
provide  information  about  the  training  evaluation  at  the  individual  critical  task  level.  The 
analysts  and  data  collectors  are  probably  the  ones  who  can  provide  the  most  valid  information 
about  the  training  environment.  They  can  document  problems  that  arose  during  training. 

In  appendix  F,  another  version  of  this  training  questionnaire  used  during  the  Maneuver  Control 
System  (MCS)  IOTE  is  presented  (18). 

2. 3. 1.2. 3  Demographic  Questionnaire.  It’s  not  clear  that  there’s  any  best  time  to  administer  the 
demographic  questionnaire — see  row  3,  column  1  of  the  first  page  of  table  4.  Part  of  the  reason 
for  administering  it  at  the  beginning  of  training  derives  from  the  understanding  that,  in 
implementing  MANPRINT,  there  is  a  lot  of  “paperwork”  to  be  completed.  If,  without 
jeopardizing  your  data  collection  effort,  forms  and  surveys  are  distributed  throughout  the  entire 
data  collection  effort  to  minimize  the  amount  of  “pain”  the  soldiers  have  to  experience  and  make 
it  easier  for  the  Test  Officer  to  accommodate  the  data  collection  intrusions  throughout  the  test. 
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CLASSROOM 

NETT  TRAINING  EVALUATION  DEBRIEF 
ASAS  BLOCK  2  OPERATIONAL  TEST 
RWS  V6.3 


Name 

PIN  no. 

(last  4  SSN) 

Please  Circle  Equipment  Used: 

HCU  VCU 

Date 

MOS  Rank 

Instructions 


Respond  for  each  task: 

1.  Most  listed  TASKS  have  performance  steps  (in  parentheses  following  TASK  NAME)  you  could  perform  in 
completing  the  TASK.  These  steps  are  shown  to  help  you  focus  on  the  aspect(s)  of  the  task  that  may  have 
been  difficult  (to  learn  or  requiring  instruction  modification). 

2.  Circle  Y  in  INSTRUCTIONAL  DIFFICULTIES  column  (Analysts)  if  there  appeared  to  be  difficulties  in 
training  this  task  by  the  instructor.  If  there  are  no  problems,  leave  uncircled. 

3.  Circle  Y  in  the  LEARNING  DIFFICULTIES  column  if  you  (the  Analyst  being  trained)  experienced  some 
difficulty  in  learning  this  task.  If  there  are  no  problems,  leave  uncircled. 

4.  If  you  think  you  know  the  cause  of  the  DIFFICULTY  (Instructional  or  Learning),  use  the  back  of  this  sheet  or 
attached  sheet  to  respond.  Indicate:  (l)TASK  NO;  (2)Particular  performance  step  creating  difficulty;  and  (3) 
your  response.  Among  the  many  possible  causes  for  the  difficulty  might  have  been:  (1)  Software/hardware 
problems,  (2)  instructor  teaching  technique/preparation,  (3)  lack  of  repetition  or  PEs,  (4)  classroom  setup, 
noise  or  distractions,  (5)  task  complexity,  (6)  manuals,  ...  Others  may  exist. 
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Table  5.  Training  evaluation  debrief  data  collection. 


Task  No. 

Task  Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0001 

Assemble  the  RWS 

Y 

Y 

RWS-0002 

Configure  the  RWS  for  Operations 

[A)Boot  RWS,  B)As  Standalone  w/wo  TOCBS, 

C)/As  Master  w/wo  TOCBS,  D)As  Client  w/wo 
TOCBS,  D)As  Client  w/wo  TOCBS] 

Y 

Y 

RWS-0003 

Initialize  the  ASAS-RWS  Software 

[SELECT:  A)Functional  Identities,  B)Time  Zones, 
C)Start  Option  Radio  Button,  D)Configuration, 
E)Initialization  Button] 

Y 

Y 

RWS-0004 

Log  on  to  The  RWS  System 

Y 

Y 

RWS-0005 

Perform  Shift  Change  Procedures 

Y 

Y 

RWS-0006 

Terminate  System  Operations 

Y 

Y 

[A)Stop-A/Control  Break] 

RWS-0007 

Disassemble  the  RWS 

Y 

Y 

RWS-0009 

Perfonn  Desktop  Tools  Menu  Operations 

[A)Set  Display  Filters,  B)Enable/Disable  Alarms, 
C)Select  Profiles,  D)Mount  Floppy  Disk/CD 
ROM/Optical  Disk,  E)Format  Floppy  Disk/Optical 
Disk] 

Y 

Y 

RWS-0010 

Perform  Printer  Management  and  Hardcopy  Services 

[A)Window,  B)Screen,  C)Window  to  file,  D)Remove 
print  job  from  queue,  E) Administer  print  labels, 
F)Display  printer  status,  G)Select  user  default  printer, 
H)Add/Delete  printer,  I)Change  IP  Address,  J)Printer 
Diagnostics] 

Y 

Y 

RWS-001 1 

Perform  File  Browser  Functions 

[A)Delete/Move/Copy/Change  Permissions, 

B)Change  Access  Control  Settings,  C)Access  Files  in 
SunPCi  Environment] 

Y 

Y 

RWS-0012 

Save  Screens  or  Windows  to  Files 

Y 

Y 

RWS-001 3 

Configure  ASAS-RWS  Nodes 

A)Add/Delete/Modify  Node  Information  (Intel  Ops 
State),  B ) Allow/Prohibit  Client  Node  Host  Automatic 
Processes 

Y 

Y 

RWS-OOl  6 

Select  Country  Codes 

[A)Search/Set  Preferences] 

Y 

Y 

RWS-0021 

Use  COE  Office  Products 

[A)Start/Shut  Down  Sun  PCi  Environment,  b)Access 
Microsoft  Word/Excel/PowerPoint 

Y 

Y 

RWS-0022 

Collaborate  Using  Sunforum 

[A)Launch  SunForum,  B)Connect  to  Work 
Station/Send  Files/Chat,  C)Share  System  Windows, 
D)Use  Whiteboard] 

Y 

Y 

RWS-0023 

Change  Nickname/Codeword 

Y 

Y 

RWS-0025 

Perform  On-Line  Operations 

[A) Access  WWW  Browser,  B)Find  and  Return  to 
Pages,  C)Open  New  Navigator 

Window/Display  Content,  D)Create  Simultaneous 
Network  Connections/New  Page,  E)Search  for 
Information] 

Y 

Y 
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Table  5.  Training  evaluation  debrief  data  collection  (continued). 


Task  No. 

Task  Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0100 

Perform  Security  Audit  Trail(SAT) 

[A)Set  Filters,  B)Collect  SAT  Data, 

C) Generate/Delete/Print  SAT  Reports, 

D)  Archive/Purge/Re  store  Files] 

Y 

Y 

RWS-0101 

Change  System  Security  Parameters 

[A)High  Water  Mark(HWM),  B)HWM  with 

Classified  Help,  C)Shift  Change  Maximum 
Time/Screen  Saver  Time] 

Y 

Y 

RWS-0102 

Maintain  User  Accounts  and  Profiles 

[A)Create/Modify/Delete/Disable,  B)Edit  Account 
Profile  Management(APM),  C)Assign/Change 
Password] 

Y 

Y 

RWS-0103 

Control  System  Process  Operations 

[A)Monitor  Process  Status,  B)Disable/Enable/Restart 
Processes] 

Y 

Y 

RWS-0200 

Perform  Comms  Message  Processor(CMP)Functions 

[A)Designate/Start/Monitor  a  CMP] 

Y 

Y 

RWS-0201 

Perform  Messaging  Using  CMP 

[A)Launch  Messaging  Tools,  B)Confirm 

Connections,  C)Set  Message  Wrapper  Defaults, 

D) Send/Edit/Review/ Archive  a  Message  with  CMP, 

E) Select/Use  Message  Filter,  F)Use  Message 

Handling  Table] 

Y 

Y 

RWS-0202 

Produce  Enemy  Interoperability  Messages(EIM) 

[A)Configure  EIM  for  Automatic/Manual  Release, 
B)Modify/Disable  a  Destination  for  EIM,  C)Generate 
EIM  from  Entity  Operations,  D)Generate  an  S507 
Resources  Logistics  Message] 

Y 

Y 

RWS-0203 

Control  Message  Release  Authority(MRA) 

[A)Enable/Disable  MRA,  B)Process  a  Message] 

Y 

Y 

RWS-0204 

Prepare  Input/Output  Media 

[A)Load  from  Input  Media,  B)Save  to  Output  Media] 

Y 

Y 

RWS-0205 

Send  a  Message  Manually 

[A)Prepare/Send(Release),  B)Add  Addressee] 

Y 

Y 

RWS-0206 

Process  an  Inbound  Message 

[A)Interactive  Message  Parsing,  B)Text  Message 
Processing,  C)Message  Criteria  Alert  Processing] 

Y 

Y 

RWS-0207 

Manage  Files  Using  X-FTP 

[A)Access  X-FTP,  B)Connect/Disconnect  to 
Destination,  C)File  Management] 

Y 

Y 

RWS-0208 

Perform  Address  Group  Maintenance 

[A)Create/Delete/Maintain] 

Y 

Y 

RWS-0210 

Perform  Analyst  Mail  Functions 

[A)Create  Address  Book  from  TOCBS,  B)Send  Mail] 

Y 

Y 

RWS-021 1 

Process  Anomalous  Messages 

[A)Add  Classified  Alias  for  Single  Message/Message 
Processing,  B)Recheck/Delete] 

Y 

Y 
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Table  5.  Training  evaluation  debrief  data  collection  (continued). 


Task  No. 

Task  Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0212 

Communicate  Using  Chatter 

[A)Initiate/ Answer  Chatter  Call] 

Y 

Y 

RWS-0213 

Perform  Message  Journal  Review 

Y 

Y 

RWS-0214 

Perform  Address  Maintenance 

[A)Create/Modify/Add  Address  Information, 

B) Change  Data  Transfer  Method  Preferences, 

C) Make  Global  Changes  to  System  Types,  D)Use 
Destination  Addressing] 

Y 

Y 

RWS-0301 

Monitor  The  Network  Operational  Status 

[A)Access  Tool,  B)Map  Network/Save,  C)Modify 
Network  Node  Attributes/Map  Display] 

Y 

Y 

RWS-0302 

Access  Remote  Hosts 

Y 

Y 

RWS-0305 

Modify  Network  Configuration 

[A)Network  Configuration  Management  Functions] 

Y 

Y 

RWS-0400 

Create  Alert  Criteria  Development  Sets 

[A)Criteria  Record,  B)Activate/Deactivate  Criteria 
Set/Record,  C)Update  Criteria  Record,  D)Set  Analyst 
Preferences  and  Filters  (APAF)] 

Y 

Y 

RWS-0401 

Create  Message  Criteria  Development  Sets  Using 

The  RWS 

[A)Create  Standing  Request  for  Information(SRI), 
B)Stop/Start  Message  Criteria  Processing,  C)Process 
a  SRI  Alert] 

Y 

Y 

RWS-0402 

Perform  Alert  Operations— Data  Criteria  Alerts(DCA) 

[A)Acknowledge  a  Data  Criteria  Alert,  B)Plot  an 

Entity  from  Data  Criteria  Alerts  Window, 
C)Generate/Transmit  DCA] 

Y 

Y 

RWS-0404 

Post  and  Manage  Imagery  Related  Intel  Products 

Using  ELT/4000 

[A)Create  Graphic  Image,  B) View/Manipulate  Image 
Using  ELT/4000,  C)Download  Secondary  Imagery, 
D)Display  Camera  Icon,  E)Register  Image  to  Map, 
F)Import  a  Textual  INTSUM,  G)Display  Image  Icon 
and  Its  Footprint  to  a  Map,  H)Sort  Image  File,  I)Set 
View  for  Image  File,  J)Delete  Image  File, 
K)Disseminate  SID  via  E-Mail] 

Y 

Y 

RWS-0405 

Process  Candidates  for  Specific  Correlation 

[A)Open  Specific  Correlation  Interactive  Window, 
B)Set  Display  of  Records,  C)Insert  Records  into 
ASCDB,  D)Retrieve  Candidates  for  Correlation, 

E) Forward  a  Record  for  Alternate  Processing, 

F) Remove  Candidate  from  Display,  G)Combine 
Records] 

Y 

Y 

RWS-0406 

Develop  the  Collection  Plan 

[A)Create,  B)Send/Receive  Plan/Synchronization 
Matrix,  C)Share  Collection  Plan  in  Multinode 
Environment] 

Y 

Y 

RWS-0407 

Develop  the  Intelligence  Synchronization 

Matrix(SM) 

[A)Create  SM,  B)Generate/Transmit  Multiple  Asset 
Tasking  Message(MATM)] 

Y 

Y 
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Table  5.  Training  evaluation  debrief  data  collection  (continued). 


Task  No. 

Task  Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0408 

Develop  a  Doctrinal  Template(DT) 

[A)Take  Entities  on  the  Map,  B)Plot  DT  to  Map, 
C)Create  Situational  Template  from  DT/Observations 
on  Map  and  Both  Sources,  D)Create  an  Event  and 
Decision  Support  Template] 

Y 

Y 

RWS-0410 

Perform  ASAS  Overlay  Operations 

[A)Create/Load/Open/Send/Export/Import  Overlays, 
B)Make  Overlay  Available  for  MCO] 

Y 

Y 

RWS-041 1 

Perform  CTP/Overlay  Operations 

[A)Start  the  Common  Tactical  Picture  Application, 
B)Create  a  New  Chart  Tab,  C)Set  a  Chart  Tab  to 
Archive  in  Overlay  Explorer,  D)Create  Plan/Overlay 
in  the  Overlay  Explorer,  E)Add  Objects  to 
Plan/Overlay,  F) Save/Remove/Delete  Overlay, 

G) Create  a  SA  Overlay  in  Overlay  Explorer, 

H) Create  SA  Overlay  for  Enemy/Friendly  Unit 
Information  in  Overlay  Explorer,  I)Edit  CP  Filter, 
J)Save  Changes  in  Overlay  Explorer,  K)Open  Palette, 
L)Select  Palette,  M)Create  New  Palette,  N)Access 
Symbol's  Definition, 

0)Select  Multi  Point  Symbols,  P)Create/Edit 
/Save/Delete  Symbol,  Q)Plot  Symbols  to  Map  from 
Milsym  Manager  Window,  R)Move  Single/Multiple 
Symbols,  S)Edit  Objects  on  Map,  T)Close  CTP 
Application] 

Y 

Y 

RWS-0412 

Manipulate  Battlefield  Geometry 

[A)Create  Friendly/Enemy  Control  Measure,  B)Edit 
Control  Measure,  C)Send  S20 1  Geometry  Message] 

Y 

Y 

RWS-041 3 

Maintain  Geographical  Areas 

[A)Create/Edit/Delete] 

Y 

Y 

RWS-0500 

Perform  Entity  Operations 

[A)Select  Entities  from  Map 

Window(Manually/Using  Criteria),  B)Remove  Entity 
Graphics,  C)Restore  Entity  Location,  D)Center  Map, 

E) Query  Database, 

F) Update/Insert/Edit/Delete/Associate/Combine 

Entities  in  Database,  G)Generate  Messages] 

Y 

Y 

RWS-0501 

Display  Entity  History 

Y 

Y 

RWS-0502 

Perform  Map  Operations 

[A)Launch  JMTK,  B)Save  Map  Area,  C)Recall  Map 
Area,  D)Create  JMTK  Snapshot,  E)Change  Map 
Properties,  F)Change  Map  Properties,  G)Display 

Map  Contours(Terrain/Bottom),  H)Adjust  Map 

Display  Intensity,  I)Recenter  Map  Display,  J)Display 
Center  Marker,  K)Change  Map  Display 
Scale(Zooming),  L)Display  Grid  Lines  on  Map 

Display,  M)Load  Map  Data  from  CD  ROM, 

N)Perform  Quick  Plot  Operations,  0)Perform 
Coordinate  Conversion,  P)Set  Display  Preferences] 

Y 

Y 
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Table  5.  Training  evaluation  debrief  data  collection  (continued). 


Task  No. 

Task  Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0600 

Execute  Database  Operations 

[A)Perform  Forms-Based  Database  Query, 

B)Perform  Expert  Query(SQL-Based),  C)Save 

Query,  D)Save  Results  of  Query,  E)Load  Query, 

F)Print  Matrix  Results  of  Query,  G)Print  Saved 
Query/Results,  H)Plot  Query  Results,  I)Center  Map 
to  Selected  Query  Results,  J)Update  Record, 
K)Customize  Display  of  Query  Results,  L)Convert 
Coordinates,  M)Set  Default  Map  for  Map  Operations 
from  Query,  N)Create  Entity  (Long/Short  Form), 

O) Create  Damage  Assessment/Control  Measures, 

P) Access  Geographical  Areas  and  Query,  Q)Retrieve 
Message  for  Queried  Item,  R)Generate  Messages] 

Y 

Y 

RWS-0601 

Create  Database  Entities 

Y 

Y 

RWS-0603 

Combine  Entities 

[A)Combined  Plotted  Entities  from  Map,  B)Change 
Views/Fields] 

Y 

Y 

RWS-0604 

Maintain  Enemy  Order  of  Battle  Tables 

[CREATE:  A)New  Facility  Types,  B)Facility  Alias, 
C)Equipment  Alias,  D)New  Equipment  Type,  E)New 
Force  Type,  F)New  Major  Branch  Type,  G)New  Unit 
Identification  Alias,  H)New  Unit  Number  Alias, 

I)New  Organization  Type,  J)New  Echelon  Type, 
K)New  Functional  Role  Type,  L)New  Radio  Type, 
M)New  Radio  Alias,  N)New  Radar  Type,  0)Radar 
Alias,  P)Select  Classification  Abbreviation] 

Y 

Y 

RWS-0605 

Perform  Parser  Table  Maintenance 

[MAINTAIN  TABLES:  A)Readdress,  B)Auto 

Discard,  C)Inbound  Routing,  D)Auto  Forward 

Criteria,  E)Auto  Forward  List] 

Y 

Y 

RWS-0606 

Associate  Entities 

Y 

Y 

RWS-0607 

Set  Criteria  for  Redundancy  Checking  of 

Incoming  Messages 

[CREATE:  A)New  Set  of  Search  Parameters  for 
Redundancy  Checking,  B)New  Record  for 

Redundancy  Checking,  C)New  Set  of  Rules  for 

Specific  Correlation,  D)New  Combination  Methods 
Set/Record,  ACTIVATE:  E)Set  of  Search 

Parameters,  F)Set  of  Specific  Correlation  Rules, 

G) New  Combination  Methods  Set, 

H) Change/Deactivate  Combination  Methods] 

Y 

Y 

RWS-0608 

Maintain  Country  Code  Tables 

[A)Search/Add/Modify  Country  Code] 

Y 

Y 

RWS-0609 

Query  the  Database  Via  WWW 

Y 

Y 

RWS-0610 

Navigate  the  Table  Maintenance  User  Interface 

Y 

Y 

RWS-0612 

Maintain  Message  Parsing  Tables 

[A)Maintain  Geographic  Reference  and  Message 
Datum  Tables] 

Y 

Y 
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Table  5.  Training  evaluation  debrief  data  collection  (continued). 


Task  No. 

Task  Name 

Instructional 

Difficulties 

Learning 

Difficulties 

RWS-0613 

Maintain  User  Defined  Database 

[A)Create  a  New  Table,  B)Create/Delete/Modify 
Attributes  for  a  Table,  C)Select  Existing  Value  Set, 
D)Create  New  Value  Set,  E)Publish  User  Defined 

Table,  F)Delete  User  Defined  Tables(Published/Not 
Published),  G)Modify  Privileges  in  a  Table,  H)Access 
Table  Maintenance  for  User  Defined  Databases, 

I)Input  Data  to  User  Defined  Table] 

Y 

Y 

RWS-0614 

Use  Table  Maintenance  Miscellaneous  Functions 

Y 

Y 

RWS-0615 

Establish  JCDB  Criteria 

[A)Access  JCDB  Criteria,  B)Deactivate  Auto 

Forward  of  ASCDB  Data  to  JCDB,  C)Establish 

Criteria  for  Forwarding  Data  to  JCDB] 

Y 

Y 

RWS-0616 

Perform  External  Database  Coordination  (EDC) 
Synchronization 

[A)Request  an  EDC,  B)Auto  Forward  an  EDC, 
C)Manually  Define  Criteria,  D)Send  an  EDC] 

Y 

Y 

RWS-0617 

Archive,  Purge,  and  Restore  Data 

Y 

Y 

RWS-0700 

Process  Target  Criteria(D281)  Messages 

[A)Review/Delete  Target  Criteria,  B) Associate 

TIDAT  from  Alerts,  C)Plot  Target  Criteria  Area  from 
Alerts,  D)Create  Alert  Criteria] 

Y 

Y 

RWS-0701 

Build  and  Manage  Target  Sheets 

[A)Create/Delete/Edit  Target  Sheet,  B)Create  Alert 
Criteria] 

Y 

Y 

RWS-0702 

Nominate  Targets 

[A)Nominate  Target  from  Active  Target  List 

Processing  Queue/Database  Operations] 

Y 

Y 

RWS-0703 

Maintain  Target  Parameters 

[A)Set  Target  Parameters,  B)Establish/Maintain 

Target  Number  Block] 

Y 

Y 

RWS-0704 

Process  Target  Coordination  Messages 

[A)Process  Target  Coordination  Request  from 
Alert/Directory,  B)Register  Target  Coordination 
Request  Alerts  in  Multimode] 

Y 

Y 

RWS-0705 

Manage  Targeting  Queues 

[A)Nominate  Target  from  Targeting  Queue, 

B) Save/Modify/Delete  Nominated  Entity] 

Y 

Y 

RWS-0706 

Maintain  Target  Type(Equipment/ 

Facility)  Translation  Tables 

Y 

Y 
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ACE  TRAINING  QUESTIONNAIRE 


Purpose:  This  questionnaire  is  used  to  capture  your  views  concerning  your  ACE  Functional  Identity  (FI) 

Classroom  Training.  The  data  you  provide  will  help  decision  makers  evaluate  how  effective  the  Training  program 
has  been  designed  to  effectively  train  soldiers  to  use  the  ACE.  None  of  the  data  you  give  will  be  provided  to  your 
unit  or  higher  level  chain  of  command  nor  entered  into  your  personal  fdes.  Your  responses  will  be  maintained  under 
strict  enforcement  of  Confidentiality  Public  Law. 

Instructions:  Read  each  item  carefully.  Enter  your  response  in  the  space  provided  or  circle  the  appropriate 
response.  We  would  welcome  any  written  comment  you  may  provide  in  response  to  a  particular  question  asked. 
Please  use  the  reverse  of  the  page  (if  necessary)  for  these  entries  and  indicate  item  number  for  which  you  are  giving 
written  comment. 

Rating  Scheme:  The  rating  criteria  used  in  this  questionnaire  are  defined  below.  Please  ensure  that  you  understand 
these  rating  criteria  when  answering  questions  with  a  rating  scheme.  Circle  number  which  indicates  your 
judgment— or  N/A  =  9  if  item  is  not  relevant  for  you. 

1:  Completely  Agree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  factor  being 
evaluated  cannot  be  any  better— in  condition,  design  or  composition  and  is  a  desirable  system  feature. 

2:  Strongly  Agree:  This  response  indicates  that  the  factor  being  evaluated  is  very  good  and  very  helpful  to  the 
analyst/operator. 

3:  Generally  Agree:  This  response  indicates  the  factor  being  evaluated  is  acceptable  and  help  fid  to  the 
analyst/operator. 

4:  Generally  Disagree:  This  response  indicates  the  factor  being  evaluated  is  unacceptable,  but  only  minor 

improvements  are  required  to  make  it  acceptable.  Written  comment  indicating  examples  of  how  change  should 
be  made  would  be  appropriate  when  this  rating  is  used. 

5:  Strongly  Disagree:  This  response  indicates  that  the  factor  being  evaluated  is  unacceptable  and  major 

improvements  are  required  to  make  it  acceptable.  Written  comment  indicating  examples  of  how  change  should 
be  made  would  be  appropriate  when  this  rating  is  used. 

6:  Completely  Disagree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  factor  being 

evaluated  is  unacceptable— condition,  design  or  composition  and  must  be  completely  changed  to  be  acceptable. 
Written  comment  indicating  examples  of  how  change  should  be  made  would  be  appropriate  when  this  rating  is 
used. 
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PART  I.  GENERAL  INFORMATION 


1 .  Dates  Training  Received:  (MM/DD/YY) 

2.  Course  Name/FI  Training: _ 

3.  Student  Name/Rank:  _ 

PART  II.  COURSE  MATERIAL 


4.  Information  received  during  this  training  will: 

1  =  Enhance  my  ability  to  do  my  job 

2  =  Somewhat  enhance  my  ability  to  do  my  job 

3  =  Be  of  no  assistance  in  doing  my  job 

5.  Technical  information  presented  during  this  training  was: 

1  =  Incomprehensible 

2  =  Elementary,  boring 

3  =  Comprehensible,  easily  understood 

6.  This  training  was: 


1  =  Too  long 

2  =  Too  short 

3  =  About  right 


7.  Reference  question  6:  If  any  part  of  the  classroom  instruction  was  too  long,  what  part(s)  could  be  shortened  and 
still  meet  your  training  needs? 


8.  Reference  question  6:  If  any  part  of  the  classroom  instruction  was  too  short,  what  part(s)  should  be  lengthened 
to  meet  your  training  needs? 


9.  Position  manuals  were  available  for  reference  for  all  parts  of  the  instruction  if  I  had  difficulty  executing  a 
function. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=9 

Indicate  part(s)  where  manuals  were  not  available.  _ 


10.  Appropriate  field  manuals  were  available  for  reference  for  all  parts  of  instruction  to  accomplish  analytical 
tasks  if  needed. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=9 

Indicate  part(s)  where  manuals  were  not  available.  _ 


1 1 .  Appropriate  technical  manuals  were  available  for  reference  for  all  parts  of  instruction  to  accomplish  operator 
level  maintenance  and  or  troubleshooting  tasks  when  needed  and/or  required. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=9 
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Indicate  part(s)  where  manuals  were  not  available.  _ 

12.  All  manuals  are  easy  to  understand. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=9 

Indicate  those  manuals  which  are  not.  _ 

13.  All  manuals  are  organized  so  that  information  can  be  quickly  referenced  and  the  information  found  and  used  in 
a  timely  manner. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=9 

Indicate  those  manuals  which  are  not.  _ 


PART  III.  COURSE  PRESENTATION 

14.  Majority  of  lessons  taught  were: 

1  =  Very  well  presented 

2  =  Adequately  presented 

3  =  Poorly  presented 

Indicate  those  lessons  which  were  poorly  presented. 


1 5 .  Instructor  presentations  were: 

1  =  Well  organized  and  easy  to  follow 

2  =  Unorganized;  difficult  to  follow 

3  =  Some  areas  (or  critical  tasks)  need  improvement  (please  list) 


16.  Instructor  presentations  were: 

1  =  Clear;  easy  to  understand 

2  =  Unclear;  difficult  to  understand 

3  =  Some  areas  (or  critical  tasks)  required  greater  clarification  (please  list) 


1 7 .  The  instructor  was : 

1  =  Well  prepared 

2  =  Somewhat  prepared  [please  list  areas  (or  critical  tasks)  not  prepared] 


3  =  Poorly  prepared 

4  =  Not  prepared 
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18.  The  instructor  was : 


1  =  Able  to  answer  questions  or  provide  assistance  as  needed 

2  =  Partially  able  to  answer  questions  or  provide  assistance  as  needed 

3  =  Unable  to  answer  questions  or  provide  assistance  as  needed  for  some  part(s) 

Indicate  part(s)  where  questions  could  not  be  answered.  _ 


19.  Practice  (hands  on  training)  needed  to  develop  individual  skills  proficiency  in  each  part  of  instruction  was: 

1  =  Most  sufficient 

2  =  Sufficient 

3  =  Partially  sufficient 

4  =  Insufficient 

20.  Reference  question  19:  If  hands-on  training  was  insufficient  to  build  operator  proficiency,  in  what  subject 
area(s)  (or  critical  tasks)  did  this  occur? 


2 1 .  The  instructor  used  available  training  time: 

1  =  Fully  and  effectively 

2  =  Fully  and  effectively  on  the  average 

3  =  Excessive  amount  of  time  was  lost 


PART  IV.  VALUE  OF  TRAINING  AIDS 

22.  During  this  training,  training  aids  were: 

1  =  Used  effectively 

2  =  Used  ineffectively 

3  =  Not  used  at  all,  but  needed 

4  =  Not  used  at  all,  not  needed 

5  =  Some  training  aids  need  further  work  (please  list  one  or  two  examples) 


23.  Training  aids  were  used: 

1  =  Too  often 

2  =  About  right 

3  =  Too  little 

24.  Training  aids  used  were: 

1  =  Of  good  quality 

2  =  Of  fair  quality 

3  =  Of  poor  quality 
Recommended  changes  (if  any) 
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PART  V.  TOOLS  AND  EQUIPMENT 


25.  I  experienced  no  software  malfunctions  which  would  cause  me  operational  difficulties  in  the  accomplishment 
of  intelligence  gathering,  processing,  analyzing,  and  dissemination  tasks. 

Completely  Disagree  1  2  3  4  5  6  Completely  Agree  N/A=9 

Indicate  software  malfunctions  experienced.  _ 


26.  I  experienced  no  hardware  malfunctions  which  would  cause  me  operational  difficulties  in  the  accomplishment 
of  intelligence  gathering,  processing,  analyzing,  and  dissemination  tasks. 

Completely  Disagree  1  2  3  4  5  6  Completely  Agree  N/A=9 

Indicate  hardware  malfunctions  experienced.  _ 


27.  My  workstation  was  of: 

1  =  Excellent  design 

2  =  Good  design 

3  =  Poor  design 

Please  indicate  recommendations  for  change. 


PART  VI.  TRAINING  FACILITIES/FACTORS 

28.  Classroom  was: 

1  =  Adequate  in  size 

2  =  Inadequate  in  size  (please  indicate  why) _ 


29.  Classroom  was: 

1  =  Free  of  noise  and  other  distractions 

2  =  Contained  noise  and  other  distractions  (please  indicate  sources) 


30.  Classroom  was: 

1  =  Adequately  lighted 

2  =  Not  adequately  lighted 

31.  Classroom  was: 

1  =  Adequately  ventilated 

2  =  Inadequately  ventilated 
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32.  Unit  requirements  caused  disruptions  to  my  training: 


1  =  Often 

2  =  Frequently 

3  =  Sometimes 

4  =  Never 

List  types  of  disruptions. 


33.  Instructor  to  student  ratios,  particularly  during  practical  exercises: 

1  =  Contributed  to  major  learning  difficulties 

2  =  Caused  no  learning  difficulty 

3  =  Caused  minor  learning  difficulty 

Indicate  why  (if  you  know)  there  were  learning  difficulties.  _ 


34.  Workstation  shortages,  particularly  during  hands-on  training: 

1  =  Contributed  to  an  inability  to  build  individual  skills  proficiency 

2  =  Caused  minor  problems  in  building  individual  skills  proficiency 

3  =  Did  not  interfere  with  building  individual  skills  proficiency 

4  =  There  was  no  workstation  shortage 

PART  VII.  PRACTICAL  EXERCISES/PERFORMANCE  EVALUATIONS 

35.  The  majority  of  the  practical  exercises  (PEs)  in  this  training: 

1  =  Greatly  assisted  in  building  operator  proficiency 

2  =  Moderately  assisted  in  building  operator  proficiency 

3  =  Did  not  contribute  to  building  operator  proficiency 

Indicate  those  which  did  not. _ 

4  =  There  was  no  practical  exercise(s)  during  this  block  of  instruction 

36.  The  majority  of  the  practical  exercises  in  this  block  on  instruction: 

1  =  Made  sense.  I  always  knew  what  I  was  doing  and  why.  I  could  always  relate  the  PE  to 

the  accomplishment  of  individual  tasks  at  my  unit. 

2  =  Made  little  sense.  I  never  knew  what  I  was  doing  or  why.  Could  not  always  relate  the 

PE  to  the  accomplishment  of  individual  tasks  at  my  unit. 

3  =  There  was  no  practical  exercise(s)  during  this  block  of  instruction. 

Indicate  those  which  did  not. _ 

4  =  Needed  some  explanation  by  instructor. 

37.  Performance  tests  evaluated  (Check  all  which  apply): 

1  =  What  was  taught  and  nothing  more 

2  =  More  than  what  was  taught 

3  =  My  knowledge  of  the  system  at  this  point  of  training 

4  =  My  level  of  proficiency  at  this  point  of  training 

5  =  There  was  no  performance  test(s)  administered  during  this  block  of  instruction 
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PART  VIII.  PERFORMANCE  OF  CRITICAL  TASKS-Use  TASK  No.  on  “Training  Assessment  Debrief’ 

38.  List  those  critical  tasks  taught  during  this  training  which  you  found  difficult  to  learn  and  hard  to  perform: 
Difficult  to  Learn  Hard  to  Perform 


39.  I  believe  the  following  critical  task(s)  requires  less  instruction  time  to  train. 


40.  I  believe  the  following  critical  task(s)  requires  more  instruction  time  to  train. 


41.  I  believe  the  following  critical  task(s)  will  require  continuous  practice  and  training  at  unit  level  to  prevent  skill 
decay  and/or  proficiency  loss. 


PART  IX.  OPINION,  REMARKS,  RECOMMENDATIONS 

42.  Training  received  during  this  training  was: 

1  =  Very  good 

2  =  Good 

3  =  Fair 

4  =  Poor 


43.  I  recommend  the  following  additions  to  this  training.  Why? 


44.  I  recommend  the  following  deletions  to  this  training.  Why? 
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Analysis  and  Control  Element  (ACE) 

Initial  Operational  Test  and  Evaluation  (IOTE) 

Demographic  Questionnaire 

Purpose:  This  questionnaire  is  used  to  capture  information  related  to  your  military  background,  MOS  experience, 
and  your  operational  familiarity  with  current  intelligence  gathering  systems. 


PRIVACY  ACT  STATEMENT 

Public  Law  93-573,  called  the  Privacy  Act  of  1974,  requires  that  you  be  informed  of  the  purpose  and  uses  to 
be  made  of  the  information  that  is  collected. 

The  Department  of  the  Army  may  collect  the  information  requested  under  authority  of  10  United  States  Code 
137. 

Providing  information  in  this  questionnaire  is  voluntary.  Failure  to  respond  to  any  particular  question  will 
not  result  in  any  penalty  for  the  respondent. 

The  information  collected  in  this  survey  will  be  used  to  evaluate  military  intelligence  systems  and  their 
utilization. 

The  information  will  be  used  for  research  and  analysis  only .  The  US  Army  Operational  Evaluation 
Command  has  primary  research,  analysis,  and  evaluation  responsibility. 


Instructions:  Read  each  item  carefully.  Enter  your  response  in  the  space  provided. 


Last  Name: _ 

(Please  Print) 

SSN: _ 


Date: _ 

MO  DA  YR 


_  First  Name: _  MI: 

(Please  Print) 

Approval 1 _ 

Signature 

Operator  Echelon: _ 

(Bde,  Div,  etc.) 


Functional  Identity  (ACE  Function) 


*SSN  is  requested  only  to  make  it  possible  to  obtain  your  ASVAB  Subtest  and  Composite  Scores  from  the  Defense 
Manpower  Data  Center  (DMDC)  in  Monterey,  CA.  ASVAB  data  and  other  personal  data  on  file  at  DMDC  will  be  used  to 
indicate  how  comparable  you  and  other  soldiers  participating  in  this  test  are  to  all  soldiers  who  will  be  using  ACE  when  it  is 
fielded. 
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I.  MILITARY  INFORMATION: 

1 .  Rank: _ 

2.  Primary  MOS: _ 

3.  Secondary  MOS: _ 

4.  DutyMOS/SSI: _ 

5.  Date  Graduated  from  AIT/OBC: _ / _ (MM/YR) 

6.  Date  Received  PMOS/SSI: _ / _ (MM/YR) 

7.  How  did  you  receive  your  PMOS/SSI  (Please  Circle  Response): 

1=AIT  2=0  JT  3=Reclassification  4=Promotion  5=0ther _ 

8.  Date  Entered  Active  Duty: _ / _ (MM/YR) 

9.  Unit  Assigned: _ 

10.  Date  Assigned: _ / _ (MM/YR) 

11.  ETS/ESA: _ / _ (MM/YR) 

12.  Projected  PCS  Date: _ / _ (MM/YR) 

13.  Current  Duty  Position: _ (outside  the  WFX) 

14.  Time  in  Current  Duty  Position: _ (MM) 

15.  Time  Using  Army  C2  Digital  Systems: _ (MM) 

16.  Civilian  Education  (Please  Circle  Response): 

a.  Current  b.  At  Last  Service  Enl 


1=  No  high  school  degree 
2=  High  school  diploma 
3=  GED 

4=  Some  college  (1-2  years) 

5=  Technical  school 
6=  Associate  Degree 

7=  Three  or  more  years  of  college  (no  degree) 

8=  College  degree 
9=  Graduate  degree 

17.  Gender  l=Male  2=  Female 
19.  Height  _ inches 


1=  No  high  school  degree 

2=  High  school  diploma 
3=  GED 

4=  Some  college  (1-2  years) 

5=  Technical  school 
6=  Associate  Degree 
7=  Three  or  more  years  of  college  (no 
degree) 

8=  College  degree 
9=  Graduate  degree 


18. 

Age 

yrs  months 

20. 

Weight _ 

lbs 

21.  Eyeware  (during  evaluation  session):  None _  Glasses _  Contacts 


22.  Handedness:  Right _  Left _ Ambidextrous 
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23.  Racial/Ethnic  Background 

1=  American  Indian/Alaskan  Native 
2=  Asian/Pacific  Islander 
3=  Black,  not  Hispanic  origin 
4=  Hispanic 

5=  White,  not  Hispanic  origin 

24.  Current  Injuries  (list) _ 

II.  INTELLIGENCE  SYSTEMS  EXPERIENCE:  The  following  questions  relate  to  your  automated  intelligence  processing 
systems  experience.  If  you  have  received  training  on  any  of  the  systems  cited  please  identify  the  nature  of  that  experience  in  the 
area  specified. 

1.  Type  System:  EPDS  l=Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

2.  Type  System:  TCAC  1  =  Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

3.  Type  System:  THMT  l=Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2  =  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

4.  Type  System:  MICROFIX  l=Yes  2=  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2  =  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

5.  Type  System:  GUARDRAIL  I=Yes  2=  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

6.  Type  System:  JSTARS  l=Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 
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7.  Type  System:  QUICKFIX 


1  =  Yes  2  =  No 


Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ _  (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _  _ 

8.  Type  System:  TEAMMATE  l=Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2  =  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

9.  Type  System:  TACJAM  1  =  Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2  =  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

10.  Type  System:  UAV  l=Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2  =  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

11.  Type  System:  TRAILBLAZER  l=Yes  2=  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Experience  on  System  Years/Months  (YR/MM) _ / _ 

12.  Type  System:  ASAS  1  =  Yes  2  =  No 

Type  Training:  Year  Trained: _ 

1  =  AIT  2=  NETT  3  =  OJT  4  =  Other: _ (Specify) 

Trained  on  ASAS  Subsystems:  1)  SS  2)  AS  3)  RWS  4)  ASAS  Light  5)  CCS 

6)  ACT-E  7)  C  I/HUM  [NT  8)  Trusted  Work  Stations 
9)  MASINT  10)  OSINT 

Experience  on  System  Years/Months  (YR/MM) _ / _ 


33 


III.  COMPUTER  EXPERIENCE: 

13.  How  many  months  experience  have  you  had  with  a  Personal  Computer/MAC  or  Laptop/Notebook  computer? 


14.  Confidence  in: 

Very 

Somewhat 

Not 

Confident 

Confident 

Confident 

Confident 

Using  Computers  in  General  _  _  _  _ 

Using  Personally  Owned  Computers  _  _  _  _ 

Using  Army  C2  Digital  Systems  _  _  _  _ 

Performing  Multiple  Tasks  at  Same  Time  _  _  _  _ 

15.  Average  Number  of  Hours  Per  Week  Spent  Using  a  Computer  (On/Off  Duty)  _ (hours) 

IV.  EXPERIENCE  WITH  SPECIFIC  INTELLIGENCE  FUNCTIONS: 

16.  Estimate  your  knowledge  in  the  use  of  JINTACCS/USMTF  message  formats. 

0=  0%  1=25%  2  =  50%  3  =  75%  4=100% 

17.  Please  record  the  number  of  vears/months  you  have  actually  performed  the  following  function  at  the  highest  command  level. 
If  sections  under  the  topical  headings  are  not  applicable  to  you,  circle  “2=No”  adjacent  to  the  sub- topic  area. 

COLLECTION  MANAGEMENT:  If  Function  is  Yes,  indicate:  Experience 

YY  MM 


Requirements  Management 

l=Yes 

2=No 

Command  Level: 

l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Requirements  Development 

l=Yes 

2=No 

Command  Level: 

l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Collection  Evaluation 

Analysis 

l=Yes 

2=No 

Command  Level: 

l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

MISSION  MANAGEMENT: 

Resource  Selection 

l=Yes 

2=No 

Command  Level: 

l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC  _ 

Non-Organic  Support 

Requests 

l=Yes 

2=No 

Command  Level: 

l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC  _ 

ASSET  MANAGEMENT: 

Organic  Resource  Task 

l=Yes 

2=No 

Command  Level: 

l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

_ Resource 

Status  Monitoring  l=Yes  2=No  Command  Level:  l=Bn  2=Bde  3=Div  4=Corp  5=EAC 


INTELLIGENCE  MESSAGE  PROCESSING: 


Message  Processing-IN 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Message  Processing-OUT 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Message  Preparation-OUT 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Processing 

Battle  Damage  Assessment 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Input 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

All  Source  Database 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

Order  of  Battle 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

ALL  SOURCE  PROCESSING: 

MTI  Processing 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

FTI  Processing 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp 

5=EAC 

HUMINT  Processing  l=Yes  2= 

=No  Command  Level: 

l=Bn  2=Bde  3=Div 

4=Corp  5=EAC 

New  Nodes  Processing 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div 

4=Corp  5=EAC 

Node  Change  Status 
SITUATION  DEVELOPMENT: 

Dynamic  Situation  Monitoring  l=Yes  2=No  Command  Level:  l=Bn  2=Bde  3=Div  4=Corp  5=EAC 

IPB  l=Yes  2=No  Command  Level:  l=Bn  2=Bde  3=Div  4=Corp  5=EAC 
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TARGET  DEVELOPMENT: 


T  arget  N  ominations  1  =  Y  es 

2=No 

Command  Level:  l=Bn  2= 

Bde  3= 

=Div  4=Corp  5=EAC 

Battle  Damage  Assessment  l=Yes 

ELECTRONIC  WARFARE  SUPPORT: 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

EW  Planning 

Evaluation  of  Current 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

Operations 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

Evaluation  of  MI  JI  Reports 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

OPSEC  SUPPORT: 

Risk  Assessment  1  =Y es 

2=No 

Command  Level:  l=Bn  2= 

Bde  3= 

=Div  4= 

=Corp  5=EAC 

Counter  Actions  Planning 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

REPORT  EVALUATING 

Effectiveness 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

SIGINT  ANALYSIS: 

COMINT/Traffic  Analysis 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

CRYPTO  Analysis 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

Signals  Analysis 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

COMINT/ELINT  Integration 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

ELINT/Signals  Analysis 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

ASSET  MANAGEMENT 

Organic  Resource  Tasking 

l=Yes 

2=No 

Command  Level 

l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 

Resource  Status  Monitoring 

l=Yes 

2=No 

Command  Level:  l=Bn 

2=Bde 

3=Div  4=Corp 

5=EAC 
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2. 3. 1.2. 4  MANPRINT  Debrief  (table  6).  The  form — referenced  in  row  4,  column  1  of  table  4 — 
presented  as  an  example  represents  an  iteration  of  the  form  used  for  collective  tasks  in  a  later 
LUTE  for  a  different  ASAS  subsystem.  Generally,  MANPRINT  Debrief  forms  are  used  for 
Individual — keystroke-oriented  critical  tasks,  as  well  as  Collective  critical  tasks.  Collective 
tasks  generally  involve  sequences  of  Individual  tasks  and  are  more  easily  identified  as  Mission- 
oriented  MOS-based  tasks.  This  particular  form  (see  page  37)  provides  more  detail  about 
problems  that  may  have  existed  in  the  training  process  that  could  be  attributed  to  the  disparity  in 
the  way  tasks  are  performed  with  and  without  computer  augmentation.  Originally,  it  was 
planned  to  be  administered  at  the  end  of  each  test  day  in  the  Pilot  and  Record  test  for  analysts 
and  data  collectors.  The  data  collectors  received  the  same  training  as  the  analysts  and  they  stood 
by  the  analysts  during  the  test  and  watched  what  they  were  doing.  The  MANPRINT  Debrief  was 
for  them  to  document  the  individual  critical  tasks  that  were  performed  and  to  indicate  those  tasks 
that  were  problematic.  While  this  form  was  designed  to  be  used  stand-alone  with  appropriate 
comments  written  on  the  form,  it  was  used  more  as  a  tabulation  sheet  for  soldiers’  referral  so  that 
they  knew  those  individual  and  collective  tasks  on  which  a  more  complete  MANPRINT 
Evaluation  was  required — on  the  next  fonn  discussed. 

2. 3. 1.2. 5  MANPRINT  Evaluation.  The  form — referenced  in  row  5,  column  1  on  the  first  page  of 
table  4 — is  used  in  conjunction  with  or  after  the  MANPRINT  Debrief.  The  MANPRINT 
Debriefs  are  forms  that  contains  key  items  that  can  apply  to  each  of  the  critical  tasks  and  asks  the 
respondents  to  provide  detailed  feedback  about  problems  in  their  performance  (or  what  they 
observed).  They  use  the  completed  fonn  (shown  in  table  6)  as  a  reference  for  detennining  the 
number  of  MANPRINT  evaluation  forms  they  are  expected  to  complete  (see  page  44).  Those 
forms  were  used  to  provide  detailed  ratings  and  comments  on  problematic  tasks. 

In  appendix  G,  a  standalone  Workload  Questionnaire  used  during  the  MCS  IOTE  is  presented 
(18). 

2. 3. 1.2. 6  Human  Factors  Engineering  General  Software  Functioning.  The  particular  version  of 
this  form  presented  here — referenced  in  row  1,  column  1  of  the  second  page  of  table  4 — was 
modified  for  use  in  the  block  2  IOTE  testing  for  ASAS  (see  page  48).  It  was  adapted  from  a 
similar  instrument  used  in  earlier  ASAS  sub-system  LUTE  testing.  Items  included  on  this  form 
addressed  the  adequacy  of  different  characteristics  of  the:  (1)  video  display;  (2)  keyboard;  and 
(3)  mechanics  of  accessing  software.  This  form  was  generally  completed  near  the  end  of  the  test 
so  that  soldiers  would  have  had  maximal  opportunity  to  make  judgments  about  the  video, 
keyboard  and  accessing  software. 
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MANPRINT  DEBRIEF  FOR 
ASAS-LIGHT  BLOCK  2  LUTE 
(COLLECTIVE  TASKS) 


BIO/BACKGROUND 

Name  PIN  no. 

DATE 

Please  Print 

Mo/Da/Yr 

Position:  ASAS  Light  Analyst 

Echelon  (please  circle): 

BTN  DIV  BDE  CORP  EAC 

INSTRUCTIONS  FOR  COLLECTIVE  CRITICAL  TASKS 


1.  Each  listed  TASK  has  sub-tasks,  many  of  which  are  completed  when  the  TASK  is  performed 
(columns  A  and  B). 

2.  For  each  TASK  (and  sub-TASK),  circle  Y  in  the  PERFORMED  column  (column  C),  if  performed  during 
collective  training. 

3.  Circle  Y  in  the  PROBLEMS  column  (column  D)  if  you  experienced  one  or  more  problems  in  performing  this 
TASK  (or  sub-TASK). 

4.  In  column  E  (1),  circle  Y  if  your  training  indicated  which  individual  tasks  you  needed  to  use  to  perform  the 
TASK  (or  sub-TASK). 

5.  In  column  E  (2),  indicate  the  ADEQUACY  of  training  you  received  in  HOW  TO  USE  those  INDIVIDUAL 
TASKS  to  complete  each  Collective  Task  (and  sub-Task)  using  the  following  scale: 

1  =  More  than  adequate 

2  =  Adequate 

3  =  Not  quite  adequate 

4  =  Barely  adequate 

5  =  Not  adequate 
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Table  6.  MANPRINT  debrief  for  collective  tasks. 


A 

Task 

No. 

B 

Task  Name 
(Collective) 

C 

Performed 

D 

Problems 

E 

Training  Identified  Individual 
Tasks  Needed  to  Perform 

Which 

Tasks 

(1) 

How  to  Use  Tasks 
(Adequacy) 

(2) 

2003 

Conduct  Intelligence 

Preparation  of  the  Battlefield 
(IPB)  (MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Establish  coordination  with 
other  staff  elements  IAW 
unit  tactical  SOP 

Y 

Y 

Y 

Y 

-B 

Define  the  battlefield  area 

Y 

Y 

Y 

Y 

-C 

Analyze/describe 
terrain/weather  effects  on 
friendly  and  enemy  COAs 

Y 

Y 

Y 

Y 

-D 

Prepare  a  modified 
combined  obstacle  overlay 

Y 

Y 

Y 

Y 

-E 

Create/update  threat 
doctrine  or  patterns  of 
operation  to  doctrinal 
templates 

Y 

Y 

Y 

Y 

-F 

Identify  threats  likely 
objectives/end  state  one  and 
two  levels  of  threat 
command  below  your  own 

Y 

Y 

Y 

Y 

-G 

Assist  in  production  of 
decision  support  template 
through  wargaming  and 
other  developed  IPB 
products 

Y 

Y 

Y 

Y 

-H 

Confirm/deny/update 
existing  estimate  of  enemy's 
COA 

Y 

Y 

Y 

Y 

2004 

Produce  Intelligence  Products 
(MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Record  incoming 
information  and  intelligence 

Y 

Y 

Y 

Y 

-B 

Provide  timely  intelligence 
support  to  targeting 

Y 

Y 

Y 

Y 

-C 

Dynamically  establish/ 
terminate  sensor-to-shooter 
links 

Y 

Y 

Y 

Y 

-D 

Produce  the  intelligence 
annex  to  the  battalion 

OPORD 

Y 

Y 

Y 

Y 

-E 

Process/disseminate  the 
Intelligence  Annex 

Y 

Y 

Y 

Y 

-F 

Produce  an  intelligence 
summary  addressing 
commander's  PIR  and  IR 

Y 

Y 

Y 

Y 
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Table  6.  MANPRINT  debrief  for  collective  tasks  (continued). 


A 

Task 

No. 

B 

Task  Name 
(Collective) 

C 

Performed 

D 

Problems 

E 

Training  Identified  Individual 
Tasks  Needed  to  Perform 

Which 

Tasks 

(1) 

How  to  Use  Tasks 
(Adequacy) 

(2) 

2005 

Disseminate  Combat 
Information  and  Intelligence 
(MNVRBN) 

Y 

Y 

Y 

Y 

-A 

Determine 
combat/intelligence 
information  that  requires 
dissemination 

Y 

Y 

Y 

Y 

-B 

Disseminate  combat 
information  to  addresses 
listed  in  tactical  SOP 

Y 

Y 

Y 

Y 

-C 

Disseminate  via  frequency 
modulated  /MSE,  tactical 
satellite,  tactical  internet... 

Y 

Y 

Y 

Y 

-D 

Confirm  receipt  of 

information/intelligence 

passed 

Y 

Y 

Y 

Y 

2009 

Intelligence  Operations 
(MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Monitor  current/projected 
enemy  situation  and  COA 

Y 

Y 

Y 

Y 

-B 

Recommend  changes  to  PIR 

Y 

Y 

Y 

Y 

-C 

Recommend  changes  to  IR 

Y 

Y 

Y 

Y 

-D 

Supervise  intelligence 
acquisition  tasks  by 
battalion  assets 

Y 

Y 

Y 

Y 

-E 

Monitor  status  of 
intelligence  information 
requests 

Y 

Y 

Y 

Y 

-F 

Initiate  new  intelligence 
information  requests 

Y 

Y 

Y 

Y 

-G 

Supervise  transfer  of 
captured  personnel/ 
documents/  material  IAW 
brigades  tactical  SOP 

Y 

Y 

Y 

Y 

-H 

Coordinate  intelligence 
effort  between  main  and 
tactical  command  post 

Y 

Y 

Y 

Y 

-I 

Supervise  release/ 
dissemination  of  intelligence 
products 

Y 

Y 

Y 

Y 

2010 

Maintain  the  Current  Enemy 
Situation  (MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Merge  significant  aspects  of 
AO  and  current  enemy 
situation 

Y 

Y 

Y 

Y 

-B 

Use  PIR/IR  in  analysis  of 
current  enemy  situation 

Y 

Y 

Y 

Y 
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Table  6.  MANPRINT  debrief  for  collective  tasks  (continued). 


A 

Task 

No. 

B 

Task  Name 
(Collective) 

C 

Performed 

D 

Problems 

E 

Training  Identified  Individual 
Tasks  Needed  to  Perform 

Which 

Tasks 

(1) 

How  to  Use  Tasks 
(Adequacy) 

(2) 

-C 

Analyze/compare  current 
enemy  dispositions/ 
compositions  with  project 
action  course 

Y 

Y 

Y 

Y 

-D 

Confirm/deny  courses  of 
action/update  enemy 
situation  and  track  status  of 
LTIOV  for  each  PIR 

Y 

Y 

Y 

Y 

-E 

Maintain  current  enemy 
situation  (capabilities 
/vulnerabilities)  with 
prioritized  COA  and 
probable  future  intent 

Y 

Y 

Y 

Y 

-F 

Maintain  situation  map  to 
show  enemy  situation  and 
locations  and  indicators  of 
future  events 

Y 

Y 

Y 

Y 

2012 

Supervise  Intelligence 
Operations  (MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Serve  as  focus  for  battalions 
intelligence  support 

Y 

Y 

Y 

Y 

-B 

Conduct  Briefings  of 
assigned  elements  on 
current  situation  and  new 
taskings 

Y 

Y 

Y 

Y 

-C 

Review  taskings  for  clarity 
and  completeness 

Y 

Y 

Y 

Y 

-D 

Prioritize  intelligence 
requirements 

Y 

Y 

Y 

Y 

-E 

Supervise  efforts  in  support 
of  IPB  process 

Y 

Y 

Y 

Y 

-F 

Submit  intelligence  products 
ready  for  dissemination 

Y 

Y 

Y 

Y 

2013 

Process  Specific  Information 
Requirements  (SIR)  Data 
(MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Identify  enemy 
characteristics  that  meet 
specific  information 
requirements 

Y 

Y 

Y 

Y 

-B 

Integrate  specific 
information  requirements 
into  reconnaissance, 
intelligence,  surveillance 
and  target  acquisition  plan 

Y 

Y 

Y 

Y 

-C 

Identify  information/ 
intelligence  gaps 

Y 

Y 

Y 

Y 

-D 

When  will  enemy 
reconnaissance  elements 
move 

Y 

Y 

Y 

Y 
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Table  6.  MANPRINT  debrief  for  collective  tasks  (continued). 


A 

Task 

No. 

B 

Task  Name 
(Collective) 

Performed 

Problems 

(E) 

Training  Identified  Individual 
Tasks  Needed  to  Perform 

Which 

Tasks 

(1) 

How  to  Use  Tasks 
(Adequacy) 

(2) 

-E 

Indicate  specifics  on 
enemy  artillery  batteries 

Y 

Y 

Y 

Y 

-F 

Forward  SIR/PIR/JR  to 
brigade  S2 

Y 

Y 

Y 

Y 

-G 

Determine  intelligence 
reliability  source/agency 

Y 

Y 

Y 

Y 

-H 

Determine  intelligence 
credibility 

Y 

Y 

Y 

Y 

-I 

Compare  incoming  data 
with  intelligence  estimate 

Y 

Y 

Y 

Y 

-J 

Compare  incoming  data 
with  intelligence  products 
developed  during  IPB 
process  (intel  indicators, 
intel  workbook,  PIRs,  IRs, 
SIRs,  Situation  map) 

Y 

Y 

Y 

Y 

-K 

Determining  validity  of 
incoming  data  based  on 
preceding  standards 

Y 

Y 

Y 

Y 

-L 

Analyze  enemy 

information/intelligence 

(re: 

strengths/vulnerabilities, 
weather  terrain...) 

Y 

Y 

Y 

Y 

-M 

Provide  estimate  of 
enemy's  ability/likeliness 
to  use  NBC 

Y 

Y 

Y 

Y 

-N 

Acquire  information  to 
conduct/develop  command 
information,  operations 
security,  and  electronic 
warfare  analysis  and  a 
tactical  deception  plan 

Y 

Y 

Y 

Y 

-O 

Estimate  potential 
effectiveness  of  enemy 
smoke  and  NBC  weapons 
on  friendly  operations 

Y 

Y 

Y 

Y 

2015 

Provide  Intelligence  Support 
to  Targeting  (MNVR  BN) 

Y 

Y 

Y 

Y 

“Decide”  Function  of 

T  argeting 

-A 

Develop  high  value/payoff 
target  lists 

Y 

Y 

Y 

Y 

-B 

Develop  target  selection 
standards  and  management 
matrix 

Y 

Y 

Y 

Y 

-C 

Determine  whether 
available  assets  and  time 
constraints  will  make  a 
target  attackable 

Y 

Y 

Y 

Y 
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Table  6.  MANPRINT  debrief  for  collective  tasks  (continued). 


A 

Task 

No. 

B 

Task  Name 
(Collective) 

Performed 

Problems 

(E) 

Training  Identified  Individual 
Tasks  Needed  to  Perform 

Which 

Tasks 

(1) 

How  to  Use  Tasks 
(Adequacy) 

(2) 

-D 

Develop  attack  guidance 
matrix  from  decision 
support  template  and  time 
phase  lines 

Y 

Y 

Y 

Y 

“Detect’Tunction  of 

T  argeting 

-E 

Provide  FSO  all  targetable 
data 

Y 

Y 

Y 

Y 

-F 

Refine  reconnaissance, 
intelligence,  surveillance 
and  target  acquisition  plan 

Y 

Y 

Y 

Y 

-G 

Inform  FSO  and  brigade 
staff  of  tactical  changes  by 
enemy 

Y 

Y 

Y 

Y 

-H 

Notify  FSO  of  changes  in 
common  relevant  picture 
or  situation  map  that  effect 
targeting  plan 

Y 

Y 

Y 

Y 

-I 

Update  decision  support 
template  to  support  attack 
guidance  matrix 

Y 

Y 

Y 

Y 

“Deliver”  Function  of 

T  argeting 

-J 

Provide  targeting  team 
with  location  and  targeting 
data 

Y 

Y 

Y 

Y 

“Assess”  Function  of 

T  argeting 

-K 

Determine  which  targets 
require  BDA 

Y 

Y 

Y 

Y 

-L 

Task  assets  to  collect  BDA 

Y 

Y 

Y 

Y 

-M 

Analyze  and  horizontally 
disseminate  BDA  results 
per  battalion  SOP 

Y 

Y 

Y 

Y 

-N 

Recommend  battle  plan 
change  based  on  BDA 

Y 

Y 

Y 

Y 

2020 

Produce  a  Reconnaissance, 
Intelligence,  Surveillance 
and  Target  Acquisition 
(RISTA)  Plan  (MNVR  BN) 

Y 

Y 

Y 

Y 

-A 

Associate  RISTA 
objectives,  PIRs,  situation 
and  event  templates  to 
indicate  expected  enemy 
COA 

Y 

Y 

Y 

Y 
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Table  6.  MANPRINT  debrief  for  collective  tasks  (continued). 


A 

Task 

No. 

B 

Task  Name 
(Collective) 

Performed 

Problems 

(E) 

Training  Identified  Individual 
Tasks  Needed  to  Perform 

Which 

Tasks 

(1) 

How  to  Use  Tasks 
(Adequacy) 

(2) 

-B 

Identify  available 
collection  assets 

Y 

Y 

Y 

Y 

-C 

Prepare  intelligence 
synchronization  matrix  to 
answer  PIRs,  targeting 
requirements  and  RISTA 
objectives 

Y 

Y 

Y 

Y 

-E 

Ensure  mix  of  assets  is 
used 

Y 

Y 

Y 

Y 

-F 

Create  RISTA  overlay 
with  RISTA  matrix  to 
address  “who”,  “what”, 
“where”,  “when”  and 
“how” 

Y 

Y 

Y 

Y 

-G 

Use  RISTA  plan  to 
develop  SIRs  for  each  PIR 

Y 

Y 

Y 

Y 

2032 

Conduct  Intelligence 

Functions  for  Deployment 

Y 

Y 

Y 

Y 

-A 

Analyze  AO 

Y 

Y 

Y 

Y 

-B 

Collect  intelligence 
products  (imagery,  terrain, 
weather) 

Y 

Y 

Y 

Y 

-C 

Conduct  IPB 

Y 

Y 

Y 

Y 

-D 

Identify  intelligence 
database  gaps 

Y 

Y 

Y 

Y 

-E 

Prepare  intelligence 
estimate/develop  threat 
models 

Y 

Y 

Y 

Y 

-F 

Recommend  PIRs  and  IRs 
to  support  operation 
concept 

Y 

Y 

Y 

Y 

'Each  task  no.  is  actually  preceded  by  “34-l-”and  followed  by  “34-00MA”. 
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MANPRINT  EVALUATION 
ANALYSIS  AND  CONTROL  ELEMENT  (ACE) 
INITIAL  OPERATIONAL  TEST  AND  EVALUATION  (IOTE) 


BIO/BACKGROUND/POSITION 

Name 

PIN  no. 

Date 

Position(circle  or  indicate):  CCS 

SMART 

(Last  4  SSN)  (Mo/Da/Yr) 

Multifunction  Work  Station  RIS/ELINT  SS 

AS 

SIT/IPB 

DBM 

SIGINT  Trusted  Suite  ISR  CI/HUMINT 

OSINT 

Test  Phase:  End  of  Test 

Problem  Description 
1 .  Task  Performing  (use  task  no.): 


2.  Most  probable  problem  causes  (circle  up  to  three  most  important): 

a.  Equipment  malfunction  (appears  unrelated  to  any  soldier  action) 

b.  Training 

(1)  Poor  or  inadequate  classroom  training  on  task  (2)  Lack  of  sustainment/collective  training  on  task 

c.  Manpower  (not  enough  soldiers  to  do  job  involving  this  task) 

(1)  Need  more  analysts  to  perform  this  task  (2)  Need  more  time  to  complete  the  task 

(3)  Need  more  responsive  maintenance  support  (4)  Need  more/better  supervision 

d.  Personnel  (task  is  outside  “normal”  duties  for  my  MOS  or  skill  level) 

e.  Health  Hazard  (task  jeopardized  my  or  another  crew  member's  safety) 

f.  System  Safety  (task  performed  created  a  safety  problem  related  to  ASAS  equipment) 

g.  Human  Factors 

(1)  Problem  working  with  ACE  Functional  Identity  equipment/hardware— difficult  or  complicated  to 
work  with  or  access  when  this  task  was  performed 

(2)  Problem  with  ACE  Functional  Identity  software  when  this  task  was  performed 

(3)  Problem  with  ACE  Functional  Identity  procedures 
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3.  Problem  Description: 


4.  Problem  reported  in  (1)  Individual  Training  (Survey)  or  (2)  Collective  Training  (Survey/Interview) 

5.  Frequency  this  task  was  performed  since  formal  class  (Individual  Training)  was  completed _ (use 

number  from  following  scale) 


1  =  at  least  once  daily  3  =  at  least  once  in  the  three  weeks 

2  =  once  or  twice  a  week  4  =  not  at  all 


6.  TM  (D.O.G.)  error  contributed  to  this  problem:  YES  NO  (circle) 

If  YES,  indicate:  (a)  TM  no. /D.O.G.  (b)  page/para  or  Figure  (c)  error  description 

a)  TM/D.O.G. _ 

b)  _ 

c)  _ 


7.  Problem  Frequency/Severity  (when  task  is  being  performed): 

Severity  (circle  Roman  numeral  indicating 

Freauencv  (circle  letter  indicating  judgment) 

judgment) 

a.  Frequent  -  continuously  experienced 

I  Catastrophic  -  death  or  system's  loss 

b.  Probable  -  will  occur  frequently 

II  Critical  -  severe  injury  or  major  system 
damage 

c.  Occasional  -  will  occur  several  times 

III  Marginal  -  minor  injury  or  system  damage 

d.  Remote  -  unlikely,  but  possible 

IV  Negligible  -  less  than  minor  injury  or 

e.  Improbable  -  very  unlikely  to  occur 

system  damage 
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8.  Job  Stress  (when  task  is  being  performed): 


a)  TEMPORAL  STRESS  -  Pace  at  which  task  must 
be  performed  is  hurried  or  rushed 


b)  MENTAL  STRESS  -  Task  completion  requires 
conscious  mental  effort  (concentration) 


Circle  No. 


Circle  No. 


1  =  Not  hurried  or  rushed 


1  =  None  needed 


2  =  A  little  hurried  or  rushed 


2  =  Little  needed 


3  =  Occasionally  hurried  or  rushed 


3  =  Moderate  amount  needed 


4  =  Frequently  hurried  or  rushed 


4  =  Extensive  amount  needed 


5  =  Very  frequently  hurried  or  rushed 


If  3  or  4,  indicate  why: 


c)  PHYSICAL  STRESS  -  Task  completion  requires 
physical  effort. 


d)  PSYCHOLOGICAL  STRESS  -  Task  completion 
causes  confusion,  frustration,  or  anxiety. 


Circle  No. 


Circle  No. 


1  =  No  demand 


1  =  None 


2  =  Little  demand 


2  =  Little 


3  =  Moderate  demand 


3  =  Moderate 


4  =  Heavy  demand 


4  =  High 


5  =  Very  heavy  demand 


5  =  Intense 


If  4  or  5,  indicate  why: 


If  4  or  5,  indicate  why: 


8.  Job  Stress  (when  task  is  being  performed):  (continued) 
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9.  Mission  successfully  performed  (even  with  problem):  100%  75%  50%  25%  0% 

10.  Fix  (procedure/task  modification  needed  to  complete  mission):  YES  NO  (circle) 

11.  Recommended  fix  (if  any)  (continue  on  back  side,  if  necessary):  _ 
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HUMAN  FACTORS  ENGINEERING  (GENERAL  PC-SOFTWARE  FUNCTIONING)  QUESTIONNAIRE 
FOR  THE  ANALYSIS  AND  CONTROL  ELEMENT  (ACE) 

INITIAL  OPERATIONAL  TEST  AND  EVALUATION  (IOTE) 


Purpose:  This  questionnaire  is  used  to  capture  your  views  concerning  the  Human  Factors  Engineering  aspects  of 
ACE.  The  data  you  provide  will  help  decision  makers  evaluate  how  effective  the  ACE  has  been  designed  to  assist 
you  in  the  performance  of  your  intelligence  gathering,  processing  and  analysis  tasks. 


PRIVACY  ACT  STATEMENT 

Public  Law  03-573,  called  the  Privacy  Act  of  1974,  requires  that  you  be  informed  of  the  purpose 
and  uses  to  be  made  of  the  information  that  is  collected. 

The  Department  of  the  Army  may  collect  the  information  requested  under  authority  of  10  United 
States  Code  137. 

Providing  information  in  this  questionnaire  is  voluntary.  Failure  to  respond  to  any  particular 
question  will  not  result  in  any  penalty  for  the  respondent. 

The  information  collected  in  this  survey  will  be  used  to  evaluate  military  intelligence  systems  and 
their  utilization. 

The  information  will  be  used  for  research  and  analysis  only.  The  US  Army  Test  and  Evaluation 
Command  (ATEC)  has  primary  research,  analysis,  and  evaluation  responsibility. 


Instructions: 

1.  On  Page  2: 

a)  Check  the  Functional  Identities  on  which  you  are  serving  as  an  Operator 

b)  Indicate  your  Echelon  (e.g.,  Btn,  Bde,  Div)  of  operation  (it  is  understood  that  some  soldiers  are  operating  in  more 

than  one  functionality) 

c)  Indicate  name  of  your  workstation— keyboard  and  monitor 

2.  On  Page  3  please  read  the  descriptions  of  the  seven  possible  responses.  For  each  of  the  questions 

beginning  on  Page  4,  read  each  question  carefully  and  circle  the  response  that  appropriately  reflects  your 
opinion.  Although  not  required,  we  would  welcome  any  written  comment  you  may  wish  to  provide  in 
response  to  a  particular  question.  Please  use  the  reverse  of  the  page  for  these  entries  and  key  your 
responses  to  the  question  number. 
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BIO/BACKGROUND/POSITION 


Name _  PIN  No. _  Date  ____________ 

(Please  Print)  (Last4SSN)  MM/DD/YY 

a)  Functional  Identities  (ACE  Components)  on  which  you  have  been  serving  as  a  Test  Participant 
Operator  (please  check): 

_ISS 
_CCS 
_  Shared  SS 
_  COMINT 
_  IMINT 
_  RIS/ELINT 
_  SIGINT/MASINT 
_  CI/HUMINT 

_ OSINT 

_ AS 

_ SIT/IPB/DB 

_ TGTDEV 

_ CM/ISR 

_ Asset  Mgmnt 

_ Trusted  Suite  (TWS) 

b)  Echelon  (Btn,  Bde,  Div) _ 

c)  Workstation _  Keyboard _  Monitor _ 
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Rating  Scheme:  The  rating  criteria  used  for  the  following  questions  are  defined  below.  Please  ensure  that  you 
understand  these  rating  criteria  when  answering  questions  with  a  rating  scheme. 


1:  Completely  Agree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  item  being  evaluated 
cannot  be  any  better  designed,  written,  or  has  great  value  and  is  a  desirable  system  feature  or  function. 

2:  Strongly  Agree:  This  response  indicates  that  the  item  being  evaluated  is  very  good  and  very  helpful  to  the 
analyst/operator. 

3:  Generally  Agree:  This  response  indicates  the  item  being  evaluated  is  acceptable  and  helpful  to  the 
analyst/operator. 

4:  Generally  Disagree:  This  response  indicates  the  item  being  evaluated  is  unacceptable,  but  only  minor 
improvements  are  required  before  it  is  acceptable.  Written  comment  indicating  examples  of  how  change  should  be 
made  would  be  appropriate  when  this  rating  is  used.  Please  write  comments  on  back  of  form  and  reference  item  no. 

5:  Strongly  Disagree:  This  response  indicates  that  the  item  being  evaluated  is  unacceptable  and  major 
improvements  are  required  to  make  it  acceptable.  Written  comment  indicating  examples  of  how  change  should  be 
made  would  be  appropriate  when  this  rating  is  used.  Please  write  comments  on  back  of  form  and  reference  item  no. 

6:  Completely  Disagree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  item  being 
evaluated  is  of  unacceptable  design,  composition,  or  value  and  must  be  completely  redesigned,  rewritten,  or 
modified  to  be  acceptable.  Written  comment  indicating  examples  of  how  change  should  be  made  would  be 
appropriate  when  this  rating  is  used.  Please  write  comments  on  back  of  form  and  reference  item  no. 

0:  Unknown:  This  response  indicates  that  you  do  not  have  sufficient  knowledge  to  answer  the  question  or  you 
have  not  experienced  the  action  or  item  in  question. 
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HUMAN  FACTORS  ENGINEERING  QUESTIONNAIRE 


1  =  Completely  Agree  3  =  Generally  Agree  5  =  Strongly  Disagree 

0  = 

No  knowledge 

2  =  Strongly  Agree  4  =  Generally  Disagree  6  =  Completely  Disagree 

or  Experience 

Environmental/Physical  Concerns 

(1)  The  brightness  level  on  the  computer  screens  can  be  adjusted  to  a 

1  2 

4 

6 

0 

comfortable  level. 

J 

•J 

U 

(2)  Eye  strain  at  the  end  of  the  shift  does  not  seem  to  be  a  problem. 

1  2 

T 

T 

T 

~6~ 

0 

(a)  Video  displays  are  free  of  flicker. 

1  2 

T 

T 

T 

~6 

0 

(b)  Video  displays  allow  good  letter  discrimination. 

1  2 

3 

4 

5 

6 

0 

(c)  Video  displays  viewing  distance  is  acceptable. 

1  2 

3 

4 

5 

6 

0 

(d)  Video  displays  angle  of  view  is  acceptable. 

1  2 

3 

4 

5 

6 

0 

(e)  Location  of  video  display  is  acceptable. 

1  2 

3 

4 

5 

6 

0 

(f)  Adjustability  of  video  displays  is  acceptable. 

1  2 

3 

4 

5 

6 

0 

(3)  The  workstation  requires  a  “glare  screen”. 

1  2 

3 

4 

5 

6 

0 

(4)  Operating  the  system  components  does  not  cause  any  physical  discomfort 

1  2 

T 

T 

T 

~6 

0 

(such  as,  back  strain  and  arm  or  wrist  strain). 

(a)  Size  of  keyboard  and  controls  is  acceptable  for  effective  use. 

1  2 

T 

T 

y 

~6 

0 

(b)  Shape  of  keyboard  and  controls  is  acceptable  for  effective  use. 

1  2 

3 

4 

5 

6 

0 

(c)  Spacing  between  controls  on  keyboard  is  acceptable  for  effective  use. 

1  2 

3 

4 

5 

6 

0 

(d)  Resistance  (too  easy  to  turn  or  push  or  too  hard  to  him  or  push)  is 
acceptable  for  effective  use. 

1  2 

3 

4 

5 

6 

0 

(e)  Keyboard  and  controls  are  correctly  labeled. 

1  2 

T 

~4 

y 

~6 

0 

(f)  Keyboard  and  controls  are  the  correct  size  for  easy  visibility. 

1  2 

3 

4 

5 

6 

0 

(g)  Keyboard  and  controls  are  easy  to  understand. 

1  2 

3 

4 

5 

6 

0 

(h)  Keyboard  and  controls  do  not  have  any  unrelated  or  confusing  markings. 

1  2 

3 

4 

5 

6 

0 

(i)  Keyboard  and  controls  are  easy  to  see  and  are  clearly  visible. 

1  2 

3 

4 

5 

6 

0 

(j)  Keyboard  and  controls  are  at  the  right  angle  of  view. 

1  2 

3 

4 

5 

6 

0 

(k)  Keyboard  and  controls  are  easily  identifiable. 

1  2 

3 

4 

5 

6 

0 

(1)  Keyboard  and  critical  controls  are  within  easy  reach. 

1  2 

3 

4 

5 

6 

0 

(m)  Keyboard  and  non-critical  controls  is  acceptable. 

1  2 

3 

4 

5 

6 

0 

(n)  Keyboard  and  non-critical  controls  are  within  easy  reach. 

1  2 

3 

4 

5 

6 

0 

(o)  Keyboard  and  controls  are  functionally  grouped  together. 

1  2 

3 

4 

5 

6 

0 

(p)  Keyboard  and  control  types  are  correct  for  the  function  they  perform. 

1  2 

3 

4 

5 

6 

0 

(q)  Keyboard  and  controls  aural  controls  and  warnings  are  easy  to  hear. 

1  2 

3_ 

4_ 

_5_ 

6_ 

0 

(5)  Dust  covers  are  required  on  key  components  of  the  system  (keyboard, 

1  2 

3 

4 

5 

6 

0 

workstation). 

Software 

(6)  It  is  easy  to  delete  unneeded  data  from  the  display  screen  while  performing 

1  2 

4 

0 

procedures  or  tasks. 

j 

u 

(7)  It  is  easy  to  access  menus. 

1  2 

3_ 

4_ 

_5_ 

6_ 

0 

(8)  The  unfamiliar  terms  or  commands  are  defined  on  the  HELP  screens. 

1  2 

3_ 

4_ 

_5_ 

6_ 

0 

(9)  Help  prompts  are  always  available  in  a  display  or  are  provided  in  response 

1  2 

3 

4 

5 

6 

0 

to  a  HELP  request  to  assist  in  accomplishing  the  task. 
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1  =  Completely  Agree  3  =  Generally  Agree  5  =  Strongly  Disagree  0  =  No  knowledge 

2  =  Strongly  Agree  4  =  Generally  Disagree  6  =  Completely  Disagree  or  Experience 

(10)  The  number  and  sequence  of  steps  required  to  accomplish  a  task  are 
logical  and  follow  an  orderly  path. 

1  2  3  4  5  6  0 

(11)  All  keystrokes/actions  performed  from  one  menu  to  the  next  are 
accomplished  using  the  same  series  of  keys/commands. 

1  2  3  4  5  6  0 

(12)  All  prompts  or  messages  appear  in  the  same  position  and  are  in  the  same 
format. 

1  2  3  4  5  6  0 

(13)  At  the  completion  of  a  task,  the  computer  provides  a  prompt  or  message 
that  shows  the  actions  are  completed. 

1  2  3  4  5  6  0 

(14)  While  actions  are  being  processed,  the  computer  displays  an  “ICON” 
showing  that  the  transaction  is  in  process. 

1  2  3  4  5  6  0 

(15)  Response  to  menu  selections,  keypad  operations,  and  graphic  operations 
are  immediate. 

1  2  3  4  5  6  0 

(16)  The  cursor  location  is  easy  to  find. 

1  2  3  4  5  6  0 

(17a)  The  cursor  can  be  moved  easily  and  accurately  to  any  location  on  the 
screen  (using  the  built-in  mouse  pad). 

1  2  3  4  5  6  0 

(17b)  The  cursor  can  be  moved  easily  and  accurately  to  any  location  on  the 
screen  (using  the  external  mouse). 

1  2  3  4  5  6  0 

(18)  It  is  easy  return  to  previous  steps  in  an  operation  to  correct  an  error  or  to 
make  other  changes. 

1  2  3  4  5  6  0 

(19)  It  is  easy  to  exit  from  a  current  operation  or  function. 

1  2  3  4  5  6  0 

(20)  It  is  easy  to  cancel  changes  and  restore  the  previous  display. 

1  2  3  4  5  6  0 

(21)  Mandatory  and  optional  data  fields  are  easily  distinguished  from  each 
other. 

1  2  3  4  5  6  0 

(22)  When  an  entry  in  an  optional  field  makes  other  normally  optional  fields 
mandatory,  the  computer  provides  sufficient  prompts  to  alert  the  operator. 

1  2  3  4  5  6  0 

(23)  When  mandatory  data  entries  are  omitted,  a  “warning  message”  alerts  the 
operator  to  provide  the  required  information. 

1  2  3  4  5  6  0 

(24)  Error  messages  are  easy  to  understand. 

1  2  3  4  5  6  0 

(25)  Prompts  are  always  provided  prior  to  overwriting  or  exiting  the 
message/file  to  avoid  loss  of  data. 

1  2  3  4  5  6  0 

(26)  An  UNDO  command  is  available  to  reverse  actions. 

1  2  3  4  5  6  0 

(27)  Menus  can  be  arranged  so  they  do  not  interfere  with  the  accomplishment 
of  tasks. 

1  2  3  4  5  6  0 

(28)  Frequently  accessed  menus  are  consistent  in  format  and  functionality  from 
one  to  another. 

1  2  3  4  5  6  0 

(29)  It  is  easy  to  correct  message  errors  character  by  character,  line  by  line,  and 
field  by  field. 

1  2  3  4  5  6  0 
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1  =  Completely  Agree  3  =  Generally  Agree  5  =  Strongly  Disagree  0  =  No  knowledge 

2  =  Strongly  Agree  4  =  Generally  Disagree  6  =  Completely  Disagree  or  Experience 

(30)  Data  fields  are  grouped  and  ordered  logically  way  which  made  message 
building  easy. 

1  2  3  4  5  6  0 

(31)  All  data  field  labels  are  easily  understood. 

1  2  3  4  5  6  0 

(32)  All  data  field  instructions  are  easily  understood. 

1  2  3  4  5  6  0 

(33)  It  was  easy  to  return  to  the  main  menu  or  base-level  functions. 

1  2  3  4  5  6  0 

(34)  Menu  options  defined  the  function  to  be  performed. 

1  2  3  4  5  6  0 

(35)  The  menu  structure  and  design  permits  immediate  access  to  critical  or 
frequently  selected  options. 

1  2  3  4  5  6  0 

(36)  The  current  position  within  the  menu  structure  was  always  known. 

1  2  3  4  5  6  0 

(37)  Menu  options  use  familiar  terminology  and  each  item  is  distinct  from  the 
other. 

1  2  3  4  5  6  0 
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2. 3. 1.2. 7  Human  Factors  Engineering  (Function-Specific)  Questionnaire  (table  11).  This 
instrument — referenced  in  row  2,  column  1  on  the  second  page  of  table  4 — was  designed  with 
the  assistance  of  New  Systems  Training  Integration  Office  (NSTIO)  staff  from  Fort  Huachuca 
and  is  designed  to  address  function-specific  software  problems.  The  specific  fonn  presented 
here  (see  page  55)  was  used  to  support  the  Block  2  IOTE  testing  for  ASAS.  Development  of  this 
instrument  utilized  the  Enabling  and  Terminal  Learning  Objectives  found  in  the  Lesson  Plans 
used  to  train  soldiers  in  their  specific  functionalities. 

Appendix  H  provides  a  rather  comprehensive  Human  Factors  Engineering  (Function-Specific) 
Questionnaire/Interview  instrument  used  during  the  Block  2  ASAS  IOTE  in  March  2005.  This 
instrument  asks  questions  for  each  of  the  participating  Analysis  and  Control  Element  (ACE) 
Functional  Identities  and  judgments  about  potential  MANPRINT  problems  from  both  an 
Equipment  and  Function  perspective.  Soldiers  were  asked  to  indicate  whether  problems  existed 
and  to  provide  ratings  about  how  well  they  agreed  with  the  specific  potential  problems  that  may 
have  existed.  Ratings  are  sometimes  helpful  as  descriptive  statistics  to  tease  out  the  importance 
of  the  reported  problems;  however,  because  the  number  of  soldiers  participating  in  each  system 
function  (Functional  Identity)  is  generally  very  small,  those  ratings  rarely  are  amenable  to  any 
inferential  statistical  analyses.  While  soldiers  were  asked  to  explain  problems  by  keying  their 
written  response  to  specific  questions,  they  frequently  did  not.  In  spite  of  very  strong  repeated 
urging  of  testing  management  to  permit  follow-up  interviews  with  each  soldier  to  clarify 
unanswered  questions  (or  questions  whose  answers  were  ambiguous),  this  opportunity  was 
denied.  With  review  of  ratings  and  responses  of  each  participant  to  each  form,  interviews  can  be 
conducted  to  clarify  responses  only  to  items  evidencing  ambiguities  needing  fuller  explanation. 
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HUMAN  FACTORS  ENGINEERING  (FUNCTION-SPECIFIC)  QUESTIONNAIRE 
FOR  THE  SINGLE  SOURCE  WORK  STATIONS 
BLOCK  2-INITIAL  OPERATIONAL  TEST  AND  EVALUATION  (IOTE)2 

Purpose:  This  questionnaire  is  designed  to  capture  data  about  the  ability  of  the  soldier  to  use  the 
Single  Source  (SS)  Work  Station  to  accomplish  the  tasks  of  COMINT,  IMINT  HUMINT,  and 
OSINT  processing  and  dissemination.  The  data  you  provide  will  help  decision  makers  evaluate 
(and  correct,  if  necessary)  how  effectively  the  Single  Source  Work  Stations  and  associated 
software  have  been  designed  to  assist  and  enhance  production  and  dissemination  of  intelligence 
products.  None  of  your  individual  data  will  be  provided  to  your  chain  of  command  nor  entered 
into  your  personnel  files.  Your  responses  will  be  maintained  under  strict  enforcement  of 
Confidentiality  Public  Law. 

Instructions:  Read  each  item  carefully  and  circle  the  appropriate  numbered  response.  The 
response  must  reflect  your  perception  of  how  well  the  Single  Source  Work  Stations  have  assisted 
and  enhanced  your  ability  to  perform  COMINT,  IMINT  HUMINT,  and  OSINT  processing  and 
dissemination.  The  questionnaire  is  broad  in  scope;  therefore,  some  questions  may  not  pertain  to 
the  specific  functionality  you  are  perfonning.  In  those  cases,  circle  “N/A  =  0”.  If  you  Disagree 
with  any  statement  (ratings  4,  5,  or  6),  an  explanation  “why”  is  required.  Please  write  your 
response  on  the  back  of  this  form  and  indicate  for  which  question  you  are  making  comment. 
Items  whose  number  is  indented  are  sub-functions  or  tasks  required  to  perform  the  function 
whose  number  is  not  indented. 


BIO/BACKGROUND/POSITION 

Name 

PIN  No.  Date 

(Please  Print) 

Equipment/System  (circle  or  indicate): 

(Last  4  SSN)  MM/DD/YY 

CI/HUMINT  Work  Station 

OSINT  Work  Station 

IMINT  Work  Station 

COMINT  Work  Station 

Test  Phase:  Evaluation  Window  2— Mid  Test 

9 

“Functions,  sub-functions  and  tasks  presented  in  this  survey  have  been  provided  through  review  of  the  Enabling  and  Terminal 
Learning  Objectives  found  in  the  Lesson  Plans  used  to  train  soldiers  of  the  ACE  Functionalities.  They  have  been  provided  by  the 
New  Systems  Training  and  Integration  Office  (NSTIO),  Fort  Huachuca,  AZ. 
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The  rating  scheme  is  as  follows: 

1:  Completely  Agree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  item  being  evaluated 
cannot  be  any  better  than  it  currently  is  and  is  a  desirable  design  feature. 

2:  Strongly  Agree:  This  response  indicates  that  the  item  being  evaluated  is  very  good  and  very  helpful  to  the 
analyst/operator. 

3:  Generally  Agree:  This  response  indicates  the  item  being  evaluated  is  acceptable  and  helpful  to  the 
analyst/operator. 

4:  Generally  Disagree:  This  response  indicates  the  item  being  evaluated  is  unacceptable,  but  only  minor 
improvements  are  required  before  it  is  acceptable.  (Please  indicate  specifically  what  improvements  need  to  be 
made.) 

5:  Strongly  Disagree:  This  response  indicates  that  the  item  being  evaluated  is  unacceptable  and  major 
improvements  are  required  to  make  it  acceptable.  (Please  indicate  specifically  what  improvements  need  to  be 
made.) 

6:  Completely  Disagree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  item  being 
evaluated  is  unacceptable  and  must  be  completely  redesigned  or  rewritten  to  be  acceptable.  (Please  indicate 
specifically  what  improvements  need  to  be  made.) 

0:  Unknown:  This  response  indicates  that  you  do  not  have  sufficient  knowledge  to  answer  the  question  or  you 
have  not  experienced  the  action  or  item  in  question. 
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Based  on  your  experience,  rate  your  ability  to  perform  the  following  functions  with  the  Single  Source  Work  Station: 
For  The  OSINT  (questions  1-36) 

1. )  The  Single  Source  Work  Station  works  well  to  perform  the  OSINT  functions. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

Processing  with  OSINT  assisted  in  performing  the  following  operations: 

2. )  EMPLOYING  THE  OPEN  SOURCE  AUTOMATED  LINK  ANALYSIS  TOOL  (OSALAT)  ON  THE 
UNCLASSIFIED  OSINT  COLLECTOR  WORKSTATION  (OCW) 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

3.)  Starting  the  Local  Task  Server 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

4.)  Creating  a  Case  File 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

5. )  Opening  a  Case  File 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

6. )  Creating  a  Search  Scenario 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

7. )  Searching  for  Information  in  OSALAT 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

8. )  Reviewing  retrieved  documents  in  the  Document  Editor 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

9. )  Exporting  Case  Data 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

10.)  CONFIGURING  THE  OSINT  EXPORT  TOOL 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

11. )  Opening  the  Output  Queue 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

12. )  Opening  the  Export  Tool 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

13. )  Configuring  the  OSINT  Export  Scheduler 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

14. )  Setting  Export  Type  to  Manual  Updates 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

15.)  EMPLOYING  THE  INDIVIDUAL,  EVENTS,  AND  ORGANIZATIONS  (IE&O)  DATABASE 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

16.)  Reviewing  IE&O  Data  in  the  Entity  Editor 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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17.)  Creating  a  New  Case 


Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

18.)  Importing  External  Data  into  the  Entity  Editor 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

19.)  Creating  New  Individual 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

20.)  Adding  a  New  Attribute  to  an  Individual 
Completely  Agree  1  2  3  4  5  6 

Completely  Disagree 

N/A=0 

21.)  Adding  an  Event  to  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

22.)  Adding  a  New  Attribute  to  an  Event 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

23.)  Adding  an  Organization  to  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

24.)  Adding  a  New  Attribute  to  an  Organization 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

25.)  Adding  Equipment  to  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

26.)  Adding  a  New  Attribute  to  a  piece  of  Equipment 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

27.)  Adding  a  Facility  to  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

28.)  Adding  a  New  Attribute  to  a  Facility 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

29.)  Adding  a  Unit  to  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

30.)  Adding  a  New  Attribute  to  a  Unit 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

31.)  Adding  a  Place  to  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

32.)  Adding  a  New  Attribute  to  a  Place 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

33.)  Creating  Relationships  in  the  Case 
Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 
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34. )  Adding  Attributes  to  an  Entity 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

35. )  Publishing  from  Entity  Editor  to  TMB 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

36. )  Plotting  from  the  Entity  Editor  to  the  Map 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

For  the  IMINT  Section  (37-47) 

37.)  The  Single  Source  Work  Station  works  well  to  perform  the  IMINT  functions. 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 


Processing  with  IMINT  assisted  in  performing  the  following  operations: 

38.)  PERFORMING  IMAGERY  OPERATIONS  ON  THE  MIP 


Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

39.)  Viewing  an  Image  using  Imagery  Explorer  (IE) 

Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

40.)  Adding  annotations  to  ELT 

Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

41.)  Geo-registering  and  converting  an  image 

Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

42.)  Exporting 

an  image  to  the  JMTK  map  (Draping) 

Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

43.)  Placing  an 

Image  Icon  on 

a  map  for  display 

Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

44.)  Pushing  an  Image  from  MIP  to  the  ISS  Multimedia  Journal 

Completely  Agree 

1  2  3 

4  5  6 

Completely  Disagree 

N/A=0 

45. )  Pushing  an  Image  from  MIP  to  a  Specific  USER  (ASWS)  on  the  ISS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

46. )  Pulling  Imagery  from  ISS  to  MIP 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

47. )  Pulling  Products  from  IPL  using  JWICS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

For  the  COMINT  Section  (48-100) 

48.)  The  Single  Source  Work  Station  works  well  to  perform  the  COMINT  functions. 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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Processing  with  COMINT  assisted  in  performing  the  following  operations: 

49.)  OPERATING  THE  MASINT-AT  INTERACTIVE  CORRELATOR 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

50. )  Performing  a  PDR  query 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

51. )  Creating  a  CDR  from  PDR(s) 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

52.)  Performing  a  Candidate  Search 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

53. )  Correlating  PDR  to  CDR 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

54. )  Filtering  Data  Records 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

55. )  Merging  CDR’s 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

56. )  CONFIGURING  THE  COMINT  EXPORT  GATEWAY 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

57. )  PERFORMING  REFERENCE  DATABASE  OPERATIONS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

58. )  QUERYING  INTELLIGENCE  SHARED  SERVER  DATABASES  VIA  THE  WEB 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

59. )  Querying  database  records  on  the  ISS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

60. )  Viewing  queried  records 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

61. )  Sending  selected  data  records  to  the  Map  on  the  ISS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

62. )  MANAGING  COMINT  ALERTS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

63. )  CORRECTING  ERRED  COMINT  MESSAGES 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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64.)  Identifying  an  erred  SIG1NT  message 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

65.)  Correcting  erred  SIGINT  messages 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

66.)  Deleting  a  bad  SIGINT  message 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

67.)  Opening  the  COMINT  Export  Gateway 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

68.)  Setting  the  update  frequency 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

69.)  Configuring  the  gateway  output  queue 

Completely  Agree  1  2  3  4  5  6 

Completely  Disagree 

N/A=0 

70.)  Performing  manual  data  transfer  functions 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

71.)  Clearing  the  Export  Gateway  log 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

72.)  Exiting  the  Export  Gateway 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

73.)  MANAGING  COMINT  CORRELATION  RULE  SETS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

74.)  OPERATING  THE  COMINT  ANALYSIS  TOOLSET 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

75.)  Observing  COMINT  Case  Explorer  functions 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree 

N/A=0 

76.)  Observing  COMINT-AT  Editor  functions 

Completely  Agree  1  2  3  4  5  6 

Completely  Disagree 

N/A=0 

77.)  Creating  a  Case  folder 

Completely  Agree  1  2  3  4  5 

6 

Completely  Disagree 

N/A=0 

78.)  Using  Case  Management  features  in  the  COMINT-AT  Case  Explorer 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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79.)  Setting  field  views  in  the  COMINT-AT  Editor 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

80. )  Plotting  COMINT  data  to  the  map  from  the  COMINT-AT  Editor 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

81. )  Generating  a  U.S.  Message  Text  Formatting  (USMTF)  message  from  COMINT  data 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

82. )  Viewing  data  record  source  messages 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

83. )  Creating  a  Parsed  Data  Record  (PDR)  manually 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

84. )  Using  the  COMINT  Case  Explorer  Search  tool  to  search  for  folder  content 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

85. )  Searching  for  Network,  CDR,  or  PDR  COMINT  data  records  using  the  COMINT 

Case  Explorer  Search  tool 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

86. )  Performing  manual  correlation  functions 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

87. )  Performing  Line  of  Bearing  resolution 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

88. )  Stopping,  Starting,  and  Pausing  the  Correlator  Service 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

89. )  Sending  selected  Case  Explorer  products  to  TMB  Folders 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

90. )  Archiving  PDRs 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

91. )  Restoring  archived  PDRs 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

92. )  OPERATING  THE  COMINT-AT  GIST  TOOL 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

93. )  OPERATING  THE  COMINT-AT  CRYPTANALYSIS  TOOL 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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94.)  USING  COMINT  VISUALIZATION  TOOLS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

95. )  Viewing  network  visualization  (network  diagram) 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

96. )  Editing  network  visualization  (network  diagram) 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

97. )  Sending  a  USMTF  message  from  COMINT  visualization 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

98. )  Viewing  Order  of  Battle  visualization 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

99. )  Publishing  COMINT  visualizations  to  Microsoft  PowerPoint 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

100. )  PERFORMING  NEAR  REAL  TIME  INTELLIGENCE  OPERATIONS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

For  the  HUMINT  section  (101-138) 

101. )  The  Single  Source  Work  Station  works  well  to  perform  the  HUMINT  functions. 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

Processing  with  HUMINT  assisted  in  performing  the  following  operations: 

102. )  PERFORMING  TWS,  WEBGUARD  AND  WEB  SHIELD  OPERATIONS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

103. )  Verifying  proxy  settings  for  WebShield  operations 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

104. )  Retrieving  a  collateral  File  via  WebShield 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

105.)  QUERYING  THE  DATABASE  VIA  THE  WEB 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

106. )  Querying  database  records  on  the  ISS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

107. )  Viewing  queried  records 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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108.)  Viewing  Association,  multimedia,  or  Cross  Links  for  data  in  the  Query  Results  window 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

109.)  PERFORMING  FILE  TRANSFER  PROTOCOL  (FTP)  OPERATIONS 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

110. )  Initiating  a  new  FTP  session 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

111. )  Setting  initial  directories  for  a  selected  profile 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

112)  Setting  advanced  settings 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

113.)  CONFIGURING  CHAMS  ACE  BLOCK  II  SYSTEM  SETTINGS 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

114.)  Configuring  CHAMS  System  Settings 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

115.)  PREPARING  THE  CI/HUMINT  WORKSTATION  FOR  OPERATIONS  (CF-73  LAPTOP) 
WITHIN  THE  ACE  BLOCK  II  CONFIGURATION 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

1 16.)  Unpacking  the  laptop  components  from  the  carrying  case 


Completely  Agree  12  3 

4 

5 

6 

Completely  Disagree 

N/A=0 

117.)  Assembling  the  computer 

Completely  Agree  12  3 

4 

5 

6 

Completely  Disagree 

N/A=0 

118.)  Attaching  other  devices 

Completely  Agree  12  3 

4 

5 

6 

Completely  Disagree 

N/A=0 

119.)  Applying  power  to  the  system 

Completely  Agree  12  3  4 

5 

6 

Completely  Disagree 

N/A=0 

120.)  CONFIGURING  CHAMS  ACE  BLOCK  II  COMMUNICATIONS 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

121. )  Configuring  Windows  2000  Communications 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

122. )  Configuring  the  CHAMS  Workstation  to  the  ACE  Classified  Printer 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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123.)  Configuring  the  CHAMS  Address  Book 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

124.)  PERFORMING  CHAMS  REPORT  MANAGER  FUNCTIONS  FOR  ACE  OPERATIONS 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

125. )  Accessing  Report  Manager 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

126. )  Creating  a  CI/HUMINT  Report  for  ACE  Transfer 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

127. )  Queuing  Reports  for  Correlation 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

128. )  Sending  Reports  into  the  ISS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

129. )  Deleting  Reports  from  the  ACE  ISS 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

130. )  Editing  a  CEHUMINT  Report 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

131. )  Plotting  a  CI/HUMINT  Report 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

132.)  PREPARING  THE  ACE  CI/HUMINT  WORKSTATION  FOR  MOVEMENT 
Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

133. )  Terminating  Operations 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

134. )  Disassembling  the  system 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

135. )  Storing  the  system 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

136. )  ESTABLISHING  AN  ACE  ADDRESS  BOOK 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

137. )  CONFIGURING  THE  ISS  CHAMS  WEB  LINK  INTERFACE 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 

138. )  ESTABLISHING  ISS  DATA  LINK  USING  THE  WEB  INTERFACE 

Completely  Agree  1  2  3  4  5  6  Completely  Disagree  N/A=0 
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2. 3. 1.2. 8  Documentation  of  Inter-Operability  Difficulties— Problems  Receiving/Sending 
Different  Message  Types  Among  (Intra/Inter)  Battlefield  Functional  Areas  (BFA)  (table  7).  This 
form — referenced  in  row  3,  column  1  on  the  second  page  of  table  4,  Documentation  of  Inter- 
Operability — provides  a  rather  comprehensive  opportunity  for  soldiers  to  report  difficulties 
receiving  and  sending  different  message  types  among  the  participating  Functional  Identities  and 
for  identifying  both  intra/inter-communication  problems  among  the  BFA.  This  form  has  been 
used  in  many  of  the  AS  AS  sub-system  LUTEs  as  well  as  the  block  2  IOTE. 
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DOCUMENTATION  OF  INTEROPERABILITY  PROBLEMS 
DURING  ANALYSIS  AND  CONTROL  ELEMENT  (ACE) 
INITIAL  OPERATIONAL  TEST  AND  EVALUATION  (IOTE) 
BLOCK  2 


BIO/BACKGROUND/POSITION 

Name  PINno. 

Date 

Time 

Please  Print 

Mo/Da/Y  r 

0000-2400 

Please  circle  ACE  position  (Functional  Identity): 

CCS  SMART  Multifunction  Work  Station 

RIS/ELINT 

SS  AS 

SIT/IPB  DBM 

SIGINT  Trusted  Suite  ISR  CI/HUMINT 

Please  circle  echelon: 

BTN  DIV  BDE  CORP  EAC 

MOPP(  Please  circle):  0  12  3  4 

OSINT 

INSTRUCTIONS 

For  each  of  the  listed  MESSAGE  TYPES,  please  indicate: 

•  whether  you  had  PROBLEM  RECEIVING 

•  FROM  WHOM  was  the  message  being  sent  (if  known) 

•  BRIEF  DESCRIPTION  of  problem 

•  whether  you  had  PROBLEM  SENDING 

•  TO  WHOM  were  you  sending  the  message 

•  BRIEF  DESCRIPTION  of  problem 
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Table  7.  Documentation  of  inter-operability  difficulties — problems  receiving/sending  different  message  types  among  (intra/inter)  battlefield  functional  areas 
(BFA). 


INTEROPERABILITY  PROBLEMS 


NAME 


a. 

MESSAGE  TYPE 

b. 

PROBLEM 
RECEIVING 
(check  if 
YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

e.  PROBLEM 
SENDING 
(check  if  YES) 

f. 

TO  WHOM 
(see  below) 

g- 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

i. 

Tactical  Reports  (TACREP)  (Cl  1 1) 

2. 

Size,  Activity,  Location,  Unit,  Time,  and 
Equipment  Report  (SALUTE)  (S303) 

3. 

Intelligence  Report  (INTREP)  (Cl  10) 

4. 

Artillery  Target  Intelligence.  Target 

Criteria  Report  (ATI.TCRIT)  (D281) 

5. 

Geometry  Message  (S201) 

6. 

Imagery  Interpretation  Report  (HR) 

7. 

Reconnaissance  Exploitation  Report 
(RECCEXREP)  (Cl 01) 

8. 

Radar  Exploitation  Report  (REXREP) 

(X03 1 ) 

9. 

Intelligence  Summary  (INTSUM) 

10. 

Order  Message  (A423) 

c.  FROM  WHOM: 
f.  TO  WHOM: 


DIVISION  LEVEL: 

1)  RWS  2)  TOC  3)  MSE  4)  CNR  5 )  NTDR  6)  SINCGARS  7)  EPLRS  8)  AUTODIN  9)  TROJAN  SPIRIT  10)CGS  1 1)  UAV  12)MCS  13)  AFATDS  14)CSSCS  15)FAADC3I  16)ASAS 
ALL  SOURCE  17)  CHATS  18)IMETS  19)DTSS  20)  ACT-E  21)FBCB2  22)  AMDWS _ 


BRIGADE  LEVEL: 

23)  RWS  24)  TOC  25)  MSE  26)CNR  27)  NTDR  28)  SINCGARS  29)  EPLRS  30)  AUTODIN  31)  TROJAN  SPIRIT  32)  CGS  33)  UAV  34)  MCS  35)  AFATDS  36)  CSSCS  37)  FAADC3I 
38)  ASAS  ALL  SOURCE  39)  CHATS  40)  IMETS  41)DTSS  42)  ACT-E  43)FBCB2  44)  AMDWS _ 


Table  7.  Documentation  of  inter-operability  difficulties — problems  receiving/sending  different  message  types  among  (intra/inter)  battlefield  functional  areas 
(BFA)  (continued). 


a. 

MESSAGE  TYPE 

b. 

PROBLEM 
RECEIVING 
(check  if 
YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

e. 

PROBLEM 
SENDING 
(check  if 
YES) 

f. 

TO 

WHOM 
(see  below) 

g- 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

ii. 

Operations  Plan  or  Order  Change  Message 
(PLANORDCHG) 

12. 

Request  for  Information  (RI)  (F014) 

13. 

Response  to  Request  for  Information 
(RRI)  (F015) 

14.  Free-text  Message  (FREETEXT) 

(S302) 

15. 

Tactical  Electronic  Intelligence  Report 
(TACELINT)  (C121) 

16. 

Weather  Forecast  Message  (WXFCST) 

17. 

Multiple  Asset  Status  Report  (MASTR) 
(S304) 

18. 

External  Database  Coordination  (EDC) 
(C110M) 

19.  Variable  Message  Format  Message 
(YMF) 

c.  FROM  WHOM: 
f.  TO  WHOM: 


DIVISION  LEVEL: 

1)  RWS  2)  TOC  3)  MSE  4)  CNR  5)  NTDR  6)  SINCGARS  7)  EPLRS  8)  AUTODIN  9)  TROJAN  SPIRIT  10)CGS  11)UAV  12)  MCS  13)  AFATDS  14)  CSSCS  15)FAADC3I  16)ASAS 
ALL  SOURCE  17)  CHATS  18)IMETS  19)DTSS  20)  ACT-E  21)FBCB2  22)  AMDWS _ 


BRIGADE  LEVEL: 

23)  RWS  24)  TOC  25)  MSE  26)CNR  27)  NTDR  28)  SINCGARS  29)  EPLRS  30)  AUTODIN  3 1 )  TROJAN  SPIRIT  32)  CGS  33)UAV  34)  MCS  35)  AFATDS  36)  CSSCS  37)  FAADC3I 
38)  ASAS  ALL  SOURCE  39)  CHATS  40)  IMETS  41)DTSS  42)  ACT-E  43)  FBCB2  44)  AMDWS _ 


Table  7.  Documentation  of  inter-operability  difficulties — problems  receiving/sending  different  message  types  among  (intra/inter)  battlefield  functional  areas 
(BFA)  (continued). 


a. 

MESSAGE  TYPE 

b. 

PROBLEM 
RECEIVING 
(check  if 

YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

e. 

PROBLEM 
SENDING 
(check  if 

YES) 

f. 

TO 

WHOM 
(see  below) 

g- 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

20.  TIDAT  (S305) 

21.  ATLIEWTC  (S308) 

22 .  MSGCHGREP  (COO  1 ) 

23.  AFU.MFR  (C241) 

24.  ATLATR  (C281) 

25.  RESOURCES  (S507L) 

26.  MAER  (S301) 

27.  MATM  (X014) 

28.  STOPJAM  (S307) 

29.  OBSERVED  POSITION  REPORT 
(K05.52) 

c.  FROM  WHOM: 
f.  TO  WHOM: 


DIVISION  LEVEL: 

1)  RWS  2)  TOC  3)  MSE  4)  CNR  5)  NTDR  6)  SINCGARS  7)  EPLRS  8)  AUTODIN  9)  TROJAN  SPIRIT  10)CGS  11)UAV  12)  MCS  13)  AFATDS  14)  CSSCS  15)FAADC3I  16)ASAS 
ALL  SOURCE  17)  CHATS  18)IMETS  19)DTSS  20)  ACT-E  21)FBCB2  22)AMDWS _ 


BRIGADE  LEVEL: 

23)  RWS  24)  TOC  25)  MSE  26)CNR  27)  NTDR  28)  SINCGARS  29)  EPLRS  30)  AUTODIN  31)  TROJAN  SPIRIT  32)  CGS  33)UAV  34)  MCS  35)  AFATDS  36)  CSSCS  37)  FAADC3I 
38)  ASAS  ALL  SOURCE  39)  CHATS  40)  IMETS  41)DTSS  42)  ACT-E  43)FBCB2  44)  AMDWS _ 


Table  7.  Documentation  of  inter-operability  difficulties — problems  receiving/sending  different  message  types  among  (intra/inter)  battlefield  functional  areas 
(BFA)  (continued). 


a. 

MESSAGE  TYPE 

b. 

PROBLEM 
RECEIVING 
(check  if 

YES) 

c. 

FROM 

WHOM 

(see 

below) 

d. 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

e. 

PROBLEM 
SENDING 
(check  if 

YES) 

f. 

TO 

WHOM 

(see 

below) 

g- 

BRIEF  DESCRIPTION 
(use  reverse  side  if  necessary) 

30.  FREETEXT  (K01.1) 

31.  SPOT/SALUTE  (K04.52) 

32.  OVERLAY  (K0.52) 

33.  SCT  digital  picture  (JPEG) 

34.  RELEVANT  COMMON  PICTURE 
(MCO) 

35.  SID  (NITF) 

36.  DIGITAL  PICTURE  (NITF) 

37.  Other  (specify) 

c.  FROM  WHOM: 
f.  TO  WHOM: 


DIVISION  LEVEL: 

1)  RWS  2)  TOC  3)  MSE  4)  CNR  5)  NTDR  6)  SINCGARS  7)  EPLRS  8)AUTODIN  9)  TROJAN  SPIRIT  10)CGS  11)UAV  12)  MCS  13)  AFATDS  14)  CSSCS  15)FAADC3I  16)ASAS 
ALL  SOURCE  17)  CHATS  18)IMETS  19)DTSS  20)  ACT-E  21)  FBCB2  22)  AMDWS _ 


BRIGADE  LEVEL: 

23)  RWS  24)  TOC  25)  MSE  26)CNR  27)  NTDR  28)  SINCGARS  29)  EPLRS  30)  AUTODIN  3 1 )  TROJAN  SPIRIT  32)  CGS  33)  UAV  34)  MCS  35)  AFATDS  36)  CSSCS  37)  FAADC3I 
38)  ASAS  ALL  SOURCE  39)  CHATS  40)  IMETS  41)DTSS  42)  ACT-E  43)  FBCB2  44)  AMDWS _ 


2. 3. 1.2.9  Final  Debrief— Analyst/Data  Collector.  These  forms — referenced  in  rows  4  and  5, 
column  1  on  the  second  page  of  table  4 — provide  an  opportunity  for  analysts  and  data  collectors, 
respectively,  to  provide  a  final  documentation  of  problems  reported  on  the  system.  Questions 
asked  (see  pages  73  and  78)  tend  to  be  broad  in  nature  concerning  the  functionalities  operational 
capability  and  the  capability  of  some  of  the  major  parameters  needed  to  navigate  through  the 
software. 
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FINAL  DEBRIEF 

ASAS  SINGLE  SOURCE  OPERATOR/ANALYST  QUESTIONNAIRE 

Purpose:  This  questionnaire  is  used  to  capture  your  views  concerning  the  operational  aspects 
and  utility  of  ASAS.  The  data  you  provide  will  help  decision  makers  evaluate  how  effective  the 
ASAS  has  been  designed  to  assist  you  in  the  performance  of  your  intelligence  gathering, 
processing  and  analysis  tasks. 


PRIVACY  ACT  STATEMENT 

Public  Law  93-573,  called  the  Privacy  Act  of  1974,  requires  that  you  be  informed  of  the  purpose  and  uses  to 
be  made  of  the  information  that  is  collected. 

The  Department  of  the  Army  may  collect  the  information  requested  under  authority  of  10  United  States 
Code  137. 

Providing  information  in  this  questionnaire  is  voluntary.  Failure  to  respond  to  any  particular  question  will 
not  result  in  any  penalty  for  the  respondent. 

The  information  collected  in  this  survey  will  be  used  to  evaluate  military  intelligence  systems  and  their 
utilization. 

The  information  will  be  used  for  research  and  analysis  only.  The  US  Army  Operational  Evaluation 
Command  has  primary  research,  analysis,  and  evaluation  responsibility. 


Instructions:  First,  read  the  descriptions  of  the  seven  possible  responses  and  then  read  each 
question  carefully  and  circle  the  response  that  appropriately  reflects  your  opinion.  Although  not 
required,  we  would  welcome  any  written  comment  you  may  provide  in  response  to  a  particular 
question.  Please  use  the  reverse  of  the  page  for  these  entries  and  key  your  responses  to  the 
question  number. 


GENERAL  INFORMATION. 


Name/Rank: 


Position  Observed: 
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Rating  Scheme:  The  rating  criteria  used  for  the  following  questions  are  defined  below.  Please 
ensure  that  you  understand  these  rating  criteria  when  answering  questions  with  a  rating  scheme. 

1:  Completely  Disagree:  There  must  be  absolutely  no  doubt  when  using  this  response  that 

the  item  being  evaluated  is  of  unacceptable  design,  composition,  or  value  and  must  be 
completely  redesigned,  rewritten,  or  modified  to  be  acceptable. 

2:  Strongly  Disagree:  This  response  indicates  that  the  item  being  evaluated  is  unacceptable 

and  major  improvements  are  required  to  make  it  acceptable. 

3:  Generally  Disagree:  This  response  indicates  the  item  being  evaluated  is  unacceptable, 

but  only  minor  improvements  are  required  before  it  is  acceptable. 

4:  Generally  Agree:  This  response  indicates  the  item  being  evaluated  is  acceptable  and 

helpful  to  the  analyst/operator. 

5:  Strongly  Agree:  This  response  indicates  that  the  item  being  evaluated  is  very  good  and 

very  helpful  to  the  analyst/operator. 

6:  Completely  Agree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the 

item  being  evaluated  cannot  be  any  better  designed,  written,  or  has  great  value  and  is  a  desirable 
system  feature  or  function. 


9:  Unknown:  This  response  indicates  that  you  do  not  have  sufficient  knowledge  to  answer 

the  question  or  you  have  not  experienced  the  action  or  item  in  question. 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

SINGLE  SOURCE  OPERATOR/ ANALYST  QUESTIONNAIRE 


(1)  I  am  confident  in  my  ability  to  properly  populate,  store,  and  maintain  Single  Source  1 
databases. 

(2)  I  am  confident  in  the  accuracy  of  the  information  contained  in  the  Single  Source  1 

databases. 

(3)  ASAS  significantly  helped  me  do  my  job  as  a  COMINT  Analyst.  1 

(4)  ASAS  assisted  me  in  producing  a  COMINT  picture  of  the  battlefield  that  was  1 

complete. 

(5)  ASAS  assisted  me  in  producing  a  COMINT  picture  of  the  battlefield  that  was  1 

accurate. 

(6)  ASAS  assisted  me  in  producing  a  COMINT  picture  of  the  battlefield  that  was  timely.  1 

(7)  ASAS  significantly  helped  me  do  my  job  as  an  ELINT  Analyst.  1 

(8)  ASAS  assisted  me  in  producing  an  ELINT  picture  of  the  battlefield  that  was  1 

complete. 

(9)  ASAS  assisted  the  analyst  observed  in  producing  an  ELINT  picture  of  the  battlefield  1 

that  was  accurate. 

(10)  ASAS  assisted  me  in  producing  an  ELINT  picture  of  the  battlefield  that  was  timely.  1 

(11)  As  the  Target  Analyst,  I  was  able  to  significantly  contribute  to  the  ACE  targeting  1 

process  by  identifying  high  payoff  targets. 

(12)  The  time  to  develop,  run,  and  display  queries  against  ASAS  databases  does  not  1 

adversely  effect  analysis. 

(13)  The  alarm  process  worked  well.  1 

(14)  Single  Source  graphics  tools  enabled  me  to  display  and  evaluate  the  effects  that  1 

terrain  and  mobility  factors  may  have  on  enemy  courses  of  action. 

(15)  The  ability  to  automatically  generate  TACREPS/TACELINTS  through  the  Single  1 
Source  TACREP  generator  was  effective. 

(16)  The  Single  Source  leadership  is  fully  aware  of  the  capabilities  of  ASAS.  1 

(17)  The  Single  Source  leadership  is  aware  of  the  required  interaction  with  the  all  source  1 

leadership  to  support  ACE  operations. 


9  =  Unknown 


2  3  4  5  6  9 

2  3  4  5  6  9 

2  3  4  5  6  9 
2  3  4  5  6  9 

2  3  4  5  6  9 

2  3  4  5  6  9 
2  3  4  5  6  9 
2  3  4  5  6  9 

2  3  4  5  6  9 

2  3  4  5  6  9 
2  3  4  5  6  9 

2  3  4  5  6  9 

2  3  4  5  6  9 
2  3  4  5  6  9 

2  3  4  5  6  9 

2  3  4  5  6  9 
2  3  4  5  6  9 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


(18)  Based  on  my  observations  and  interactions  with  other  Single  Source  Analysts,  we  1  2  3  4  5  6  9 
were  able  to  effectively  contribute  to  answering  PIRs. 

(19)  ADP  operations  personnel  appeared  to  be  always  able  to  identify  the  source  of  1  2  3  4  5  6  9 

software  problem. 

(20)  ADP  operations  personnel  appeared  to  be  always  able  to  access  the  command  line  1  2  3  4  5  6  9 
and  immediately  fix  software  problems. 


(21)  Single  Source  systems  administrator  requires  access  to  system  software  through  the  1  2  3  4  5  6  9 
command  line. 


(22)  ASAS  operations  would  be  significantly  improved  if  a  designated  operator/analyst  1  2  3  4  5  6  9 
was  trained  in  UNIX  and  had  command  line  access. 


(23)  ASAS  enabled  me  to  handle  the  volume  of  messages  received  during  the  LUT. 

(24)  The  inability  to  maintain  continual  communication  with  external  systems  impacted 
on  my  ability  to  conduct  Single  Source  analysis. 

(25)  The  Single  Source  enclave  was  easily  displaced  in  MOPP  0. 

(26)  The  Single  Source  enclave  was  easily  emplaced  in  MOPP  0. 

(27)  The  Single  Source  enclave  was  easily  displaced  in  MOPP  IV. 

(28)  The  Single  Source  enclave  was  easily  emplaced  in  MOPP  IV. 

(29)  I  was  able  to  accomplish  analytical  tasks  using  ASAS  while  operating  in  MOPP  IV. 

(30)  Software  failures  during  the  tactical  phase  of  the  LUT  did  not  cause  a  loss  of 
confidence  in  the  system. 

(31)  The  majority  of  the  software  problems  can  be  traced  to  database  permissions  being 
improperly  set. 

(32)  When  a  hardware  problem  occurred,  military  maintenance  personnel  responded  in  a 
timely  manner. 

(33)  When  a  hardware  problem  occurred,  military  maintenance  personnel  isolated  the 
problem  quickly. 

(34)  When  a  hardware  problem  occurred,  military  maintenance  personnel  were  able  to 
fix  the  problem  without  difficulty. 

(35)  ASL  and  PLL  allocations  at  organic  level  appear  to  be  sufficient.  Downtime  due  to 
lack  of  spare  parts  was  not  experienced. 


1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 

1  2  3  4  5  6  9 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 


(36)  Operating  under  ACE  Concepts  and  Doctrine  enhances  the  ability  to  provide  1  2 

complete,  timely,  and  accurate  intelligence  for  tactical  decision  making  by  the  G3  or 
operational  staff. 

(37)  Sufficient  personnel  were  available  to  operate  ASAS  on  a  routine  basis.  1  2 

(38)  The  ability  to  pass  graphic  formatted  information  to  outside  the  ACE  is  critical  to  1  2 

the  utility  of  ASAS. 

(39)  The  ability  to  maintain  continuity  of  operations  during  displacement  is  a  required  1  2 

capability. 

(40)  The  basic  load  of  ASAS-unique  digital  maps  is  sufficient  to  support  contingency  1  2 

operations. 

(4 1 )  Dedicated  contractor  ADP  and  hardware  support  is  required  to  support  the  ASAS  at  12 
an  acceptable  operational  level. 

(42)  The  ability  to  exchange  databases  with  the  All  Source  enclave  and  the  CE  via  the  1  2 

EDC  message  was  effective. 

(43)  ASAS  does  not  present  any  Safety  problems.  1  2 

(44)  ASAS  does  not  present  any  Health  Hazards.  1  2 

(45)  I  was  able  to  accomplish  my  individual  critical  tasks  using  ASAS  while  operating  in  12 
MOPP  IV. 

(46)  I  was  able  to  accomplish  my  collective  critical  tasks  using  ASAS  while  operating  in  12 
MOPP  IV. 

(47)  Additional  personnel  are  not  required  to  establish  and  sustain  the  operations  of  a  12 

field  SCIF. 

(48)  ASAS  allows  me  to  edit  compartmented  information  prior  to  release  to  “protect”  1  2 

from  unauthorized  disclosure. 

(49)  ASAS  conforms  with  TEMPEST  requirements  and  no  TEMPEST  violations  were  1  2 

noted  during  ASAS  operations. 

(50)  The  operator/analyst  can  easily  use  the  ASAS  to  properly  sanitize  messages  prior  to  12 
dissemination  to  a  collateral  source. 

(51)  The  Single  Source  TEXTA  database  was  complete  and  accurate  and  a  useful  1  2 

COMINT  tool. 

(52)  The  Single  Source  EPL  database  was  complete  and  accurate  and  a  useful  EL1NT  1  2 

tool. 

(53)  The  ASAS  automated  graphics  tools  (i.e.,  building  AOIs,  boundaries,  etc.)  were  easy  1  2 
to  use. 
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FINAL  DEBRIEF 

ASAS  SINGLE  SOURCE  DATA  COLLECTOR  QUESTIONNAIRE 

Purpose:  This  questionnaire  is  used  to  capture  your  views  concerning  the  operational  aspects  and  utility  of  ASAS. 
The  data  you  provide  will  help  decision  makers  evaluate  how  effective  the  ASAS  has  been  designed  to  assist  you  in 
the  performance  of  your  intelligence  gathering,  processing  and  analysis  tasks. 


PRIVACY  ACT  STATEMENT 

Public  Law  93-573,  called  the  Privacy  Act  of  1974,  requires  that  you  be  informed  of  the  purpose  and  uses  to 
be  made  of  the  information  that  is  collected. 

The  Department  of  the  Army  may  collect  the  information  requested  under  authority  of  10  United  States  Code 
137. 

Providing  information  in  this  questionnaire  is  voluntary.  Failure  to  respond  to  any  particular  question  will 
not  result  in  any  penalty  for  the  respondent. 

The  information  collected  in  this  survey  will  be  used  to  evaluate  military  intelligence  systems  and  their 
utilization. 

The  information  will  be  used  for  research  and  analysis  only.  The  US  Army  Operational  Evaluation 
Command  has  primary  research,  analysis,  and  evaluation  responsibility. 


Instructions:  First,  read  the  descriptions  of  the  seven  possible  responses  and  then  read  each  question  carefully  and 
circle  the  response  that  appropriately  reflects  your  opinion.  Although  not  required,  we  would  welcome  any  written 
comment  you  may  provide  in  response  to  a  particular  question.  Please  use  the  reverse  of  the  page  for  these  entries 
and  key  your  responses  to  the  question  number. 


GENERAL  INFORMATION. 


Name/Rank: 


Position  Observed: 


78 


Rating  Scheme:  The  rating  criteria  used  for  the  following  questions  are  defined  below.  Please  ensure  that  you 
understand  these  rating  criteria  when  answering  questions  with  a  rating  scheme. 

1:  Completely  Disagree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  item  being 
evaluated  is  of  unacceptable  design,  composition,  or  value  and  must  be  completely  redesigned,  rewritten,  or 
modified  to  be  acceptable. 

2:  Strongly  Disagree:  This  response  indicates  that  the  item  being  evaluated  is  unacceptable  and  major 
improvements  are  required  to  make  it  acceptable. 

3:  Generally  Disagree:  This  response  indicates  the  item  being  evaluated  is  unacceptable,  but  only  minor 
improvements  are  required  before  it  is  acceptable. 

4:  Generally  Agree:  This  response  indicates  the  item  being  evaluated  is  acceptable  and  helpful  to  the 
analyst/operator. 

5:  Strongly  Agree:  This  response  indicates  that  the  item  being  evaluated  is  very  good  and  very  helpful  to  the 
analyst/operator. 

6:  Completely  Agree:  There  must  be  absolutely  no  doubt  when  using  this  response  that  the  item  being  evaluated 
cannot  be  any  better  designed,  written,  or  has  great  value  and  is  a  desirable  system  feature  or  function. 

9:  Unknown:  This  response  indicates  that  you  do  not  have  sufficient  knowledge  to  answer  the  question  or  you 
have  not  experienced  the  action  or  item  in  question. 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


SINGLE  SOURCE  DATA  COLLECTOR  QUESTIONNAIRE 

(1)  I  am  confident  in  my  ability  to  properly  populate,  store,  and  maintain  Single  Source 
databases. 

(2)  I  am  confident  in  the  accuracy  of  the  information  contained  in  the  Single  Source 
databases. 

(3)  ASAS  significantly  helped  the  analyst  observed  do  his/her  job  as  a  COMINT 
Analyst. 

(4)  ASAS  assisted  the  analyst  observed  in  producing  a  COMINT  picture  of  the 
battlefield  that  was  complete. 

(5)  ASAS  assisted  the  analyst  observed  in  producing  a  COMINT  picture  of  the 
battlefield  that  was  accurate. 

(6)  ASAS  assisted  the  analyst  observed  in  producing  a  COMINT  picture  of  the 
battlefield  that  was  timely. 

(7)  ASAS  significantly  helped  the  analyst  observed  do  his/her  job  as  an  ELINT  Analyst. 

(8)  ASAS  assisted  the  analyst  observed  in  producing  an  ELINT  picture  of  the  battlefield 
that  was  complete. 

(9)  ASAS  assisted  the  analyst  observed  in  producing  an  ELINT  picture  of  the  battlefield 
that  was  accurate. 

(10)  ASAS  assisted  the  analyst  observed  in  producing  an  ELINT  picture  of  the 
battlefield  that  was  timely. 

(11)  The  Target  Analyst  was  able  to  significantly  contribute  to  the  ACE  targeting 
process  by  identifying  high  payoff  targets. 

(12)  The  time  to  develop,  run,  and  display  queries  against  ASAS  databases  does  not 
adversely  effect  analysis. 

(13)  The  alarm  process  worked  well. 

(14)  Single  Source  graphics  tools  enabled  the  analyst  to  display  and  evaluate  the  effects 
that  terrain  and  mobility  factors  may  have  on  enemy  courses  of  action. 

(15)  The  ability  to  automatically  generate  TACREPS/TACELINTS  through  the  Single 
Source  TACREP  generator  was  effective. 

(16)  The  Single  Source  leadership  is  fully  aware  of  the  capabilities  of  ASAS. 

(17)  The  Single  Source  leadership  is  aware  of  the  required  interaction  with  the  all  source 
leadership  to  support  ACE  operations. 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 


(18)  Based  on  my  observations,  the  interactions  with  other  Single  Source  Analysts  1  2 

enabled  them  to  effectively  contribute  to  answering  PIRs. 

(19)  ADP  operations  personnel  appeared  to  be  always  able  to  identify  the  source  of  1  2 

software  problem. 

(20)  ADP  operations  personnel  appeared  to  be  always  able  to  access  the  command  line  1  2 
and  immediately  fix  software  problems. 

(21)  Single  Source  systems  administrator  requires  access  to  system  software  through  the  1  2 
command  line. 

(22)  ASAS  operations  would  be  significantly  improved  if  a  designated  operator/analyst  1  2 
was  trained  in  UNIX  and  had  command  line  access. 

(23)  ASAS  enabled  the  analyst  observed  to  handle  the  volume  of  messages  received  1  2 

during  the  LUT. 

(24)  The  inability  to  maintain  continual  communication  with  external  systems  impacted  1  2 
on  the  ability  to  conduct  Single  Source  analysis. 

(25)  The  Single  Source  enclave  was  easily  displaced  in  MOPP  0.  12 

(26)  The  Single  Source  enclave  was  easily  emplaced  in  MOPP  0.  12 

(27)  The  Single  Source  enclave  was  easily  displaced  in  MOPP  IV.  1  2 

(28)  The  Single  Source  enclave  was  easily  emplaced  in  MOPP  IV.  1  2 

(29)  The  analyst  observed  was  able  to  accomplish  analytical  tasks  using  ASAS  while  1  2 

operating  in  MOPP  IV. 

(30)  Software  failures  during  the  tactical  phase  of  the  LUT  did  not  cause  a  loss  of  12 

confidence  in  the  system. 

(31)  The  majority  of  the  software  problems  can  be  traced  to  database  permissions  being  1  2 
improperly  set. 

(32)  When  a  hardware  problem  occurred,  military  maintenance  personnel  responded  in  a  12 
timely  manner. 

(33)  When  a  hardware  problem  occurred,  military  maintenance  personnel  isolated  the  1  2 

problem  quickly. 

(34)  When  a  hardware  problem  occurred,  military  maintenance  personnel  were  able  to  12 
fix  the  problem  without  difficulty. 

(35)  ASL  and  PLL  allocations  at  organic  level  appear  to  be  sufficient.  Downtime  due  to  1  2 
lack  of  spare  parts  was  not  experienced. 

(36)  Operating  under  ACE  Concepts  and  Doctrine  enhances  the  ability  to  provide  1  2 

complete,  timely,  and  accurate  intelligence  for  tactical  decision  making  by  the  G3  or 
operational  staff. 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 


(37)  Sufficient  personnel  were  available  to  operate  ASAS  on  a  routine  basis.  1  2 

(38)  The  ability  to  pass  graphic  formatted  information  to  outside  the  ACE  is  critical  to  1  2 

the  utility  of  ASAS. 

(39)  The  ability  to  maintain  continuity  of  operations  during  displacement  is  a  required  1  2 

capability. 

(40)  The  basic  load  of  ASAS-unique  digital  maps  is  sufficient  to  support  contingency  1  2 

operations. 

(4 1 )  Dedicated  contractor  ADP  and  hardware  support  is  required  to  support  the  ASAS  at  12 
an  acceptable  operational  level. 

(43)  ASAS  does  not  present  any  Safety  problems.  1  2 

(44)  ASAS  does  not  present  any  Health  Hazards.  1  2 

(45)  The  Operator/ Analyst  was  able  to  accomplish  the  individual  critical  tasks  using  1  2 

ASAS  while  operating  in  MOPP  IV. 

(46)  The  Operator/ Analyst  was  able  to  accomplish  the  collective  critical  tasks  using  1  2 

ASAS  while  operating  in  MOPP  IV. 

(47)  Additional  personnel  are  not  required  to  establish  and  sustain  the  operations  of  a  12 

field  SCIF. 

(48)  ASAS  allows  the  Operator/Analyst  to  edit  compartmented  information  prior  to  12 

release  to  “protect”  from  unauthorized  disclosure. 

(49)  ASAS  conforms  with  TEMPEST  requirements  and  no  TEMPEST  violations  were  1  2 
noted  during  ASAS  operations. 

(50)  The  Operator/ Analyst  can  easily  use  the  ASAS  to  properly  sanitize  messages  prior  to  12 
dissemination  to  a  collateral  source. 

(51)  The  Single  Source  TEXTA  database  was  complete  and  accurate  and  a  useful  1  2 

COMINT  tool 

(53)  The  ASAS  automated  graphics  tools  (i.e.,  building  AOIs,  boundaries,  etc.)  were  easy  1  2 
to  use. 
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2.3.2  Use  of  Model  2  for  MANPRINT  Evaluation  of  the  Heavy  Equipment  Transporter 
System  (HETS) 

The  model  designated  as  model  2  appears  to  work  well  with  soldiers  who  have  relatively  low 
verbal  skill  like  those  who  operate  and  maintain  the  Heavy  Equipment  Transporter  System 
(HETS)  and  when  the  critical  tasks  have  been  identified. 

2.3.2. 1  Resources  Used  for  MANPRINT  Evaluation  of  HETS.  In  this  test,  there  were  about 
50  players  who  were  responsible  for  operating  and  maintaining  the  system.  In  order  to  collect 
sufficient  Reliability  Availability  and  Maintainability  (RAM)  data  on  the  HETS  equipment,  the 
test  took  6  months  to  complete.  The  MANPRINT  Manager  developed  the  data  collection  plan, 
designed  the  data  collection  instruments  and  directed  the  data  collection  process  with  two 
MANPRINT  support  personnel. 

2. 3. 2. 1.1  Acquiring  a  Contractor  to  Support  MANPRINT  Evaluation.  When  assigned  to  the 
Heavy  Equipment  Transporter  System  (HETS)  test  as  MANPRINT  Manager,  the  Test  Officer 
indicated  that  one  of  their  contract  data  collectors  could  be  dedicated  to  support  the  MANPRINT 
effort.  The  one  selected  had  system  knowledge — he  knew  automotive  equipment,  had  rebuilt 
vehicles,  and  had  data  processing  skills.  He  had  been  a  U.S.  Army  Aviation  Warrant  Officer 
who  was  riffed  out  of  the  service.  Following  selection  of  this  individual  to  support  the 
MANPRINT  effort,  the  data  collection  and  analysis  effort  was  scoped  out  in  greater  detail. 

2. 3. 2. 1.2  Acquiring  a  TEMP-Hire  to  Support  MANPRINT  Evaluation.  As  the  data  collection 
process  was  progressing,  Operational  Evaluation  Command  (OEC),  now  Army  Evaluation 
Command  (AEC),  indicated  that  they  didn’t  think  there  was  enough  support  for  the  MANPRINT 
effort;  accordingly  the  MANPRINT  effort  was  supplemented  with  support  from  a  government 
TEMP-hire  worker.  As  the  MANPRINT  data  collection  process  was  already  under  way,  it 
seemed  appropriate  to  begin  by  letting  the  TEMP-hire  work  with  the  original  contractor  data 
collector  to  get  him  “read-in.”  Conceptually,  the  plan  seemed  appropriate,  but  being  new  to 
supervisory  responsibilities,  it  was  not  understood  that  contract  personnel  cannot  direct/supervise 
government  employees.  Early-on  the  arrangement  seemed  workable,  but  the  contract  data 
collector  began  to  be  more  directive  rather  than  advisory  when  the  MANPRINT  Manager  was 
not  present.  When  the  TEMP  hire  documented  this  conflict  in  writing  and  pointed  out  the 
inappropriate  supervisory  role  that  had  emerged,  it  was  again  necessary  to  re-scope  the  effort. 

2. 3. 2.2  Data  Collection  Procedures  for  MANPRINT  Evaluation  of  the  HETS.  The  basic  data 
collection  strategy  for  this  effort  involved  conducting  a  series  of  individual  interviews  with  each 
of  the  system  operators  (drivers  and  assistant  drivers)  and  system  maintainers. 

2. 3. 2. 2.1  MANPRINT  Interview  Protocol  Used  With  HETS  Operators  and  Maintainers.  The 
interviews  involved  use  of  a  common  set  of  questions  comprised  of  about  four  pages  for  each  of 
the  operational  and  maintenance  critical  tasks  keyed  to  each  of  the  MANPRINT  domains 
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(see  page  85).  The  set  of  interview  questions  for  each  of  the  operator  critical  tasks  were  bound 
together  in  a  booklet;  a  comparable  booklet  was  compiled  for  system  maintainers. 

23.2.2.2  Emerging  Concerns  for  a  HETS  Task.  To  support  the  interview  process,  a  booklet 
containing  sheets  documenting  the  emerging  concerns  associated  with  performing  each  task  was 
provided  for  reference  and  as  a  possible  memory  jogger  for  each  interviewee  (see  page  89).  As 
each  interview  was  conducted,  new  concerns  addressed  by  the  interviewee  were  added  to  this 
list. 

2.3.2.23  Technical  Manual  Procedure  for  Performing  Task  Being  Interviewed.  During 
discussion  of  the  interview  procedure  with  the  Training  Systems  Manager  (TSM),  he  expressed 
concern  that  the  interviews  were  being  conducted  within  a  shelter  rather  than  outside  where  the 
Ml 070  cab  and  Ml 000  semi-trailer  were  located.  His  concern  was  that  when  a  problem  is  noted, 
ambiguity  about  the  equipment  being  discussed  could  arise.  To  address  this  concern  and  to 
provide  a  comfortable  environment  for  the  soldiers  to  fonnulate  their  thoughts,  the  interview 
process  was  facilitated  by  use  of  tabbed  pages  from  the  appropriate  Technical  Manuals  (TM)  that 
showed  step  by  step  procedures  (with  illustrations)  for  conducting  the  task  currently  under 
discussion  (table  17,  see  page  90).  During  the  interview  process,  the  interviewer  usually  sat 
opposite  the  soldier  and  the  booklets  containing  the  emerging  list  of  concerns  and  the  TM  could 
be  easily  rotated  on  the  table  so  that  the  interviewee  could  be  very  specific  about  his  problem 
descriptions. 

23.2.2.4  Supplementary  MANPRINT Data  Collection.  Conflict  between  Contract  and  TEMP- 
Hire  cited  in  section  2.3.2. 1.2,  in  part,  stimulated  the  need  to  re-scope  the  MANPRINT 
Evaluation  effort.  Emerging  findings  from  early  interviews  had  indicated  that  there  were 
concerns  about  temperature  and  noise  in  and  around  the  HETS  cab.  As  the  first  set  of  interviews 
had  been  pretty  much  completed  by  this  time,  the  TEMP  hire  was  directed  to  get  some  hard  data 
to  determine  whether  there  really  was  a  temperature  and  noise  problem.  In  order  to  collect  this 
data,  the  TEMP  hire  was  provided  with  a  sound  pressure  meter  and  thermometer  and  instructed 
to  make  arrangements  to  go  out  with  the  drivers.  He  was  given  an  experimental  design  for 
collecting  temperature  and  noise  data.  Results  of  that  data  collection  effort  are  presented  in 
appendix  I.  Studying  the  results  tables  and  the  accompanying  figure  should  provide  sufficient 
information  for  discerning  that  design.  Presentation  of  this  description  indicates  the  multiplier 
effect  of  providing  additional  support  to  the  MANPRINT  data  collection  effort.  Had  the 
contractor  and  TEMP  support  not  been  provided,  the  MANPRINT  manager  would  probably  have 
had  to  be  content  with  completing  a  single  set  of  interviews  (conducted  by  himself)  involving 
each  HETS  operator  and  maintainer. 
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HETS  MODEL  2  USED  WITH  LIST  OF  CONCERNS  AND  TABBED  TMS 


17. 

1. 

2. 


TASK:  Load  a  Disabled  Payload  on  a  M1Q70  Tractor  and  a  M1000  Semi-trailer  Combination  Using  Dual 
Winches  (-14  pp.  2-125  to  2-130) 

Did  you  perform  or  observe  this  task  being  performed?  (Y  N  ) 

If  No,  go  to  sheet  for  next  TASK. 

Problems? 

a.  Have  you  had  any  problems  in  performing  this  task?  (Y  N  ) 

1)  If  No,  go  to  2b. 

2)  If  Yes,  please  describe: _ 


b.  Are  you  aware  of  problems  others  have  had  in  performing  this  task? 
(Y  N  )  [refer  to  sheet  of  concerns] 

1)  If  No  to  2a  and  2b,  to  6. 

2)  If  Yes,  please  describe: _ 


3.  Which  concerns  are  most  important  (up  to  3)? 


Indicate  if  there  are  other  problems  which  you  do  not  see  on  the  sheet  of  concerns: 


Problem  Frequency  (when  ~  task  is  being  performed)  Circle  letter  indicating  judgment. 


CONCERN 

1st 

2nd 

3rd 

A 

A 

A  Frequent  —  continuously  experienced 

B 

B 

B  Probable  —  will  occur  several  times 

C 

C 

C  Occasional  —  will  occur  several  times 

D 

D 

D  Remote  —  unlikely  but  possible 

E 

E 

E  Improbable  —  very  unlikely  to  occur 

Problem  Severity: 

Circle  Roman  numeral  indicating  judgment. 

CONCERN 

1st 

2nd 

3rd 

I 

I 

I 

Catastrophic  —  death  or  systems  loss 

II 

II 

II 

Critical  --  severe  injury  or  major  system  damage 

III 

III 

III 

Marginal  —  minor  injury  or  minor  system  damage 

IV 

IV 

IV 

Negligible  —  less  than  minor  injury  or  system  damage 
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4. 


If  Yes  to  2,  could  one  or  more  of  these  problems  have  been  avoided  by  using: 


a.  More  people?  (  Y  N  ) 

1)  If  Yes,  how  many  more? 

b.  A  different  procedure?  (  Y  N  )  If  Yes,  please  describe: 
1 )  What  is  wrong  with  current  procedure? 


2)  What  changes  can  you  recommend? 


3)  What  modification  of  the  procedure  might  make  the  task  “doable”  with  only  two  soldiers? 


c.  An  equipment  modification  (  Y  N  ) 

1)  Please  describe  your  recommendations: 


d.  More  lecture  training?  (  Y  N  ) 

e.  More  hands-on  training?  (  Y  N  ) 

f.  Another  training  procedure?  (  Y  N  ) 
1)  If  Yes,  please  describe. 


g.  Greater  caution?  (  Y  N  ) 

1)  If  Yes,  does  performing  this  task  create  a  safety  hazard  (for  the  soldier  or  equipment)? 
(  Y  N  ) 

2)  If  Yes,  please  describe. 
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5. 


Job  Stress 


a.  Please  indicate  on  the  following  scale  the  extent  this  task  must  be  performed  at  the  same  (or  nearly  the 
same)  time  as  other  tasks  (TIME  STRESS) 

1  =  No  overlap 

2  =  Little  overlap 

3  =  Occasional  overlap 

4  =  Frequent  overlap 

5  =  Very  frequent  overlap 

If  4  or  5,  which  other  task(s)?  _ 

b.  Please  indicate  on  the  following  scale  the  extent  conscious  mental  effort  (or  concentration)  is  required 
to  perform  the  task  (MENTAL  STRESS) 

1  =  None  needed 

2  =  Little  needed 

3  =  Moderate  amount  needed 

(a)  Why? _ 


4  =  Extensive  amount  needed 
(a)  Why? _ 


c.  Please  indicate  on  the  following  scale  the  extent  performing  this  task  requires  physical  effort 
(PHYSICAL  STRESS). 

1  =  No  demand 

2  =  Little  demand 

3  =  Moderate  demand 

4  =  Heavy  demand 

5  =  Very  heavy  demand 

(a)  Why? _ 


d.  Please  indicate  on  the  following  scale  the  extent  performing  this  task  causes  (for  Operators:  you  to 
feel)  (for  Data  Collectors:  you  to  observe  signs  of)  confusion,  frustration,  or  anxiety  (PSYCHOLOGICAL  STRESS) 

1  =  None 

2  =  Little 

3  =  Moderate 

4  =  High 

a)  Why? _ 


5  =  Intense 

(a)  Why? 
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6.  Manual  Description  of  Procedure  for  Performing  This  Task  (turn  to  pages  in  TM  for  this  task) 

(Y  =  Acceptable  N  =  Changes  Needed) 

a.  Are  you  aware  of  anything  which  is  not  technically  correct  in  the: 

1)  Written  procedure  (  Y  N  ) 

(a)  If  Yes,  describe  problem. 


2)  Figures  or  drawings  (  Y  N  ) 

(a)  If  Yes,  describe  problem. 


b.  Are  you  aware  of  anything  which  is  not  understandable  in  the: 


1) 


why? 


Written  procedure  (  Y  N  ) 
(a)  If  Yes, 


2) 


why? 


Figures  or  drawings  (  Y  N  ) 
(a)  If  Yes. 
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CONCERNS  ASSOCIATED  WITH 

17.  TASK:  Load  a  disabled  payload  on  a  M1070  tractor  and  a  M1000 

semi-trailer  combination  using  dual  winches 

9.  (S)  WINCE!  OPERATION  —  Can't  see  cables,  must  use  spotter. 

10.  (S)  ENGINE  KILL  SWITCH  —  Required,  PTO  is  in  CAB,  too  far. 

15.  PTO  SWITCH/LIGHT  —  Spacing  too  far,  relocate. 


21.  (S)  CHAINING  A  TANK  —  Must  reach,  arms  extended,  with  chains  (30 — 100  lbs)  can  present  potential 

back  injury  situation.  Females  only  65%  (average)  of  males  upper  body  strength.  Must  crawl  under  load  to  rig  it. 


23.  PAYLOAD  CHOCKS  —  Replace  nuts  and  bolts  with  quick  release  pins. 


29.  (S)  LOADING,  WINCHING,  SECURING  LOAD  —  Load  old  system  in  15  minutes,  this  one  takes  an 

hour.  Can't  see  winch  cables,  tie  downs  are  not  user  friendly,  must  climb  under  load  to  secure  it,  must  count  chain 
links. 


31.  TIE  DOWNS  —  Not  enough,  relocate,  strength,  no.  of  shackles. 
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Figure  1.  Technical  manual  procedure  for  performing  task  being  interviewed. 


UNVALIDATED  COPY 


TM  9-2330-381-14 


(27)  Restow  all  tools  used  during  loading  procedure. 


Z-29.  pa&L  witfca  LOADlmS/'tMl/i&filHS 

a.  Dual  Winch  Loading. 

(1)  To  couple  tractor,  position  chocks,  curb  guides  and  chains,  and 
adjust  platform  and  ramps,  perform  steps  (1)  thru  (21)  of  paragraph 
2-2 6a. 


WARMING 


Always  wear  leather  gloves  when  handling  rope.  Never  allow  rope  to  run 
through  hands  or  serious  personal  injury  may  result. 

MOTE 

Manila  rope  will  be  used  to  pull  winch  cable  through  snatch  block  for  use 
during  unloading  procedures. 
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Figure  1.  Technical  manual  procedure  for  performing  task  being  interviewed  (continued). 


TM  9-2330-381-14 


(INVALIDATED  COPY 


(2)  Remove  maul. la  rope  (1)  from  semitrailer  storage  compartment. 
Starting  from  front  of  trailer,  pass  one  end  of  manlls  rope  (1) 
through  snatch  block  (2)  back  to  front  of  trailer. 

(3)  Secure  both  ends  of  nanlla  rope  (1)  to  lifting  eyes  on  platform. 


WABMINC 

Observe  the  following  precautions  during  the  loading  process: 

If  possible,  provide  ample  clear  space  behind  the  disabled  payload 
during  loading  to  protect  personnel  and  prevent  equipment  damage 
should  cables  break  while  payload  la  being  loaded. 

All  ground  personnel  must  stand  clear  of  loading  cables. 

Hake  sure  winch  cables  are  not  kinked  and  all  blocks  and  shackles 
are  in  good  condition  and  properly  secured. 

Hake  sure  winch  cablea  are  inspected  In  accordance  with  TB  43-0142 
or  serious  personal  Injury  may  result. 

Extreme  caution  should  be  exercised  during  any  operation  on  a  slope. 

Two  ground  spotters  must  stand  off  each  rear  corner  of  the 
semitrailer  and  maintain  visual  contact  with  the  winch  operator. 

The  spotters  must  observe  cables,  snatch  blocks,  shackles,  and 
payload  position  during  loading. 
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Figure  1.  Technical  manual  procedure  for  performing  task  being  interviewed  (continued). 


{INVALIDATED  COPY 


TM  9-2330-381-14 


WARNING 


During  winch-on  operations  on  a  downgrade,  the  payload  must  be 
restrained  from  the  rear  with  some  other  vehicle  to  prevent  possible 
loss  of  control  of  the  payload. 

Do  not  overload  towing  vehicle  winches.  Know  the  ratings  of  the 
winches  being  used  and  any  protection  devices  (such  as  shear  pins), 
or  serious  personal  injury  or  death  may  result. 

At  no  tine  during  any  loading  operations  should  personnel  be  on  the 
semitrailer  bed  or  serious  Injury  to  personnel  and  daaage  to 
equipment  may  result. 

Always  wear  leather  gloves  when  handling  cable.  Never  allow  cable 
to  run  through  hands  or  serious  injury  may  result. 

(4)  Unhook  winch  cables  from  stowage  point  on  towing  vehicle.  Winch 
operator  pay  out  enough  winch  cables  (3)  to  pass  through  both 
gooseneck  fair leads  (4). 

(3)  Cross  winch  cables  (3)  and  attach  clevis  end  of  winch  cables  to  upper 
recovery  eyes  (S)  on  payload. 


(6)  Station  ground  spotters  on  each  side  of  payload  vehicle  to  provide 
directions  to  winch  operator  during  winching  operation. 
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Figure  1.  Technical  manual  procedure  for  performing  task  being  interviewed  (continued). 


TO  9-2330-381-14 


UkVALIDATED  COPY 


2-27.  DUAL  WINCH  LOADINC/USLOADINS  (COtlfT 

CAPTION 

If  the  payload  does  not  have  a  working  track,  use  boards  or  sheets 
of  wood  to  protect  the  semitrailer  loading  ramp  and  beavertall  from 
the  payload  towing  shackles  or  lifting  eyes,  or  damage  to  the 
semitrailer  and  excess  strain  on  the  winches  may  result. 


(7)  Using  winches,  winch  operator  pulls  payload  Into  alinement  with 
ramps.  Slowly  pull  payload  up  ramps  onto  platform,  keeping  both 
winch  cables  (3)  under  tension  at  all  times.  ■  Continue  to  winch 
payload  onto  platform. 


HETS-2  4JB 


CAUTION 

When  payload  road  wheels  are  over  the  semitrailer  front  bogies,  the 
cables  must  be  uncrossed  or  damage  to  the  gooseneck  or  cables  may 
result. 


Figure  1.  Technical  manual  procedure  for  performing  task  being  interviewed  (continued). 
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Figure  1.  Technical  manual  procedure  for  performing  task  being  interviewed  (continued). 


UNVALIDATED  COPY 


TM  9-2330-381-14 


DO  HOT  disconnect  winch  cables  until  platform  is  level  and  payload  is 

chocked  or  personal  injury  may  result. 

(9)  Chock  rear  of  payload  using  two  rear  payload  chocks  (6).  Level 
platform  (para.  2-19). 

(10)  Remove  winch  cables  (3)  from  payload  upper  recovery  eyes  (5). 
Uncross  winch  cables  (3)  and  reconnect  cables  for  a  straight  pull. 

(11)  Winch  operator  continues  winching  until  payload  makes  firm  contact 
with  front  payload  chocks  (7).  Keep  tension  on  payload  until 
payload  can  be  chocked. 

(12)  To  secure  payload,  perform  steps  (28)  thru  (37)  of  paragraph  2-26a. 

(13)  Restow  winch  cables  and  winching  materials. 
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2.3.3  Use  of  Model  2  for  MANPRINT  Evaluation  of  the  Enhanced  Position  Location 
Reporting  System  (EPLRS) 

Model  2  was  also  employed  for  MANPRINT  Evaluation  of  the  Enhanced  Position  Location 
Reporting  System  (EPLRS)  IOTE  (see  footnote  c,  table  3,  page  1 1).  The  model  appeared  to  work 
reasonably  well  with  General  Purpose  User  (GPU)  soldiers  with  varying  levels  of  verbal  skill. 

2.3.3. 1  Resources  Used  for  MANPRINT  Evaluation  of  EPLRS.  Players  in  this  test  included 
about  200  soldiers  with  a  variety  of  MOSs  and  skill  levels.  The  test  was  conducted  in  three 
phases  during  a  period  of  one  month.  MANPRINT  support  was  provided  by  a  GS-12  Manager 
and  three  GS  5  TEMPS — one  of  whom  was  experienced  in  supporting  MANPRINT  evaluations 
for  other  systems  under  tests.  She  served  as  the  “in  the  field”  data  manager  who  provided  day  to 
day  direction  to  other  TEMPS  and  guidance  to  four  military  data  collectors.  The  MANPRINT 
Manager  developed  the  Data  Collection  Plan,  designed  the  data  collection  instruments  and 
coordinated  with  Test  Management  Team  to  assure  that  agreed  upon  arrangement  for  data 
collection  remained  in  place.  The  GS  5s  and  supporting  military  collected  and  processed  the 
data  and  provided  the  raw  data  and  frequency  tabulations  to  the  MANPRINT  Manager. 

2. 3. 3.2  Data  Collection  Procedures  for  MANPRINT  Evaluation  of  EPLRS.  In  contrast  to  the 
HETS  test  where  there  were  only  about  50  soldier  participants  and  a  long  test  window 

(6  months),  the  EPLRS  test  involved  about  200  soldier  participants  for  a  30-day  test. 
Consequently  interviews  had  to  be  more  targeted  to  those  soldiers  who  had  problems  on  specific 
tasks.  All  participating  soldiers  completed  one  of  seven  forms  like  that  shown  on  page  97 — 
containing  different  sets  of  tasks  for  the  different  operator  types.  In  each  case  they  checked 
those  critical  tasks  on  which  they  experienced  one  or  more  problems — and  for  which  more 
detailed  MANPRINT  data  appeared  necessary.  The  second  part  of  this  data  collection  effort  (see 
page  98)  involved  use  of  an  a  MANPRINT  Evaluation  form  that  contained  interview  questions 
paralleling  those  used  for  the  HETS  operators’  and  maintainers'  interviews  (see  page  15). 
Appendix  J  provides  a  slightly  different  version  of  the  MANPRINT  Biographical  Questionnaire 
used  for  this  system  evaluation  that  includes  additional  biographical  information  and  addresses 
potentially  important  social  attitudinal  factors. 
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Part  1 


MANPRINT  DEBRIEF  FOR  NCS  OPERATORS 
BIO/BACKGROUND/POSITION 


Name _ PIN  No. 

Please  circle: 


Date _ 

Mo/Da/Yr 


Test  Phase:  Pilot  Stand-alone  Integrated 

MOPP:  0  12  3  4 

INSTRUCTIONS 
RESPOND  FOR  EACH  TASK: 

1.  Circle  Y  in  PROBLEM  column  if  completion  of  the  task  created  a  performance  problem  for  you  since  you  last 
completed  this  form. 

IF  TEST  PHASE  IS  PILOT,  CIRCLE  Y  FOR  PROBLEMS  WHICH  OCCURRED  ONLY  AFTER  PILOT  PHASE 

BEGAN 

2.  For  those  cases  where  you  circled  Y  in  the  PROBLEM  column,  if  you  believe  your  problem  was  caused  by  a 
training  deficiency,  circle  Y  in  the  TRAINING  column. 


Task  No. 

Task  Name 

Problem 

Training 

Related 

1 

List  History,  Need,  Purpose  and  Key  Concepts  of  EPLRS 

Y 

Y 

2 

List  Major  EPLRS  Assemblies  and  Match  Purpose  of  Each 

Y 

Y 

3 

List  NCS  Controls  and  Indicators  and  Match  Purpose  of  Each 

Y 

Y 

4 

List  EPLRS  Safety,  ESD  and  Security  Requirement 

Y 

Y 

5 

Perform  System  Power-On  Procedure 

Y 

Y 

6 

Perform  Key  Load  Operations 

Y 

Y 

7 

Perform  Program  Load  Procedures 

Y 

Y 

8 

Perform  Initialization  Procedures 

Y 

Y 

9 

Perform  System  Power-Down  Procedure 

Y 

Y 

10 

Enter,  Update,  Delete  and  Record  Library  Data 

Y 

Y 

11 

Enter,  Update,  Delete  and  Record  Map  Data 

Y 

Y 

12 

Perform  Basic  Operator  Control  Switch  Actions 

Y 

Y 

13 

Perform  Map  Tape  Building/Updating  Switch  Actions 

Y 

Y 

14 

Perform  Display  Manipulation  Switch  Actions 

Y 

Y 

15 

Perform  Map  Scale/Offset  Switch  Actions 

Y 

Y 

16 

Enter  System  Configuration  Parameters 

Y 

Y 

17 

Resolve  Alerts  Advisory 

Y 

Y 
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Part  2 


MANPRINT  ASSESSMENT 


BIO/BACKGROUND/POSITION 


Name 


PIN  No. _ Date _ Time _ 

Mo/Da/Yr  0000-2400 


Please  circle: 

Position:  NCS  Oper  RS  Oper  NCS  Maint  RS  Maint  EGRU/Gateway  Oper  ETS  DS  Maint  SYSCON 
Test  Phase:  ETE  Pilot  Stand-alone  Integrated 
MOPP:  0  12  3  4 


PROBLEM  DESCRIPTION 

1.  Task  Performing  (use  task  no.  from  Critical  Task  List  when  possible): 


2.  Problem  Description: 


3.  Most  probable  problem  cause  (circle  one): 

a.  Equipment  malfunction  (appears  unrelated  to  any  soldier  action) 

b.  Training 

1)  Poor  or  inadequate  classroom  training  on  task 

2)  Lack  of  sustainment/collective  training  on  task 

c.  Manpower  (not  enough  soldiers  to  do  job  involving  this  task) 

d.  Personnel  (task  is  outside  “normal”  duties  for  my  MOS  or  skill  level) 

e.  Health  Hazard  (task  jeopardized  my  or  another  crew  member's  safety) 

f.  System  Safety  (task  performed  created  a  safety  problem  related  to  EPLRS  equipment) 
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g.  Human  Factors 

1 )  Problem  working  with  EPLRS  equipment/hardware— difficult  or  complicated 
to  work  with  or  access  when  this  task  was  performed 

2)  Problem  with  EPLRS  software  when  this  task  was  performed 

4.  Problem  reported  previously:  YES  NO  (circle) 

If  YES,  go  to  item  9;  otherwise  continue  with  item  5. 

5.  Frequency  this  task  was  performed  since  class  instruction— BEFORE  PILOT  TEST  BEGAN  (use  number  from 

following  scale) _ 


1  =  at  least  once  daily 

4  =  at  least  once  a  month 

2  =  once  or  twice  a  week 

5  =  not  at  all 

3  =  at  least  every  two  weeks 

6.  TM  error  contributed  to  this  problem:  YES  NO  (circle) 

If  YES  indicate:  a)  TMno.  b)  page/para  or  Figure  c)  error  description 

a)  TM  No. _ 

b)  _ 

c)  _ 


7.  Problem  Frequency/Severity  (when  task  is  being  performed): 

Frequency  (circle  letter  indicating  judgment) 

Severity  (circle  Roman  numeral  indicating  judgment) 

I  Catastrophic  -  death  or  system's  loss 

a.  Frequent  -  continuously  experienced 

II  Critical  -  severe  injury  or  major  system  damage 

b.  Probable  -  will  occur  frequently 

III  Marginal  -  minor  injury  or  system  damage 

c.  Occasional  -  will  occur  several  times 

IV  Negligible  -  less  than  minor  injury  or  system 

d.  Remote  -  unlikely,  but  possible 

damage 

e.  Improbable  -  very  unlikely  to  occur 
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8.  Job  Stress  (when  task  is  being  performed): 

a)  TIME  STRESS  -  Task  must  be  performed  at  or 
nearly  at  same  time  as  other  tasks. 

b)  MENTAL  STRESS  -  Task  completion  requires 
conscious  mental  effort  (concentration) 

Circle  No. 

Circle  No. 

1  =  No  overlap 

1  =  None  needed 

2  =  Little  overlap 

2  =  Little  needed 

3  =  Occasional  overlap 

3  =  Moderate  amount  needed 

4  =  Freauent  overlap 

4  =  Extensive  amount  needed 

5  =  Very  frequent  overlap 

If  3  or  4,  indicate  why: 

If  4  or  5,  which  other  tasks  (use  nos.  from  Critical 

Task  List,  if  possible) 

c)  PHYSICAL  STRESS  -  Task  completion  requires 
physical  effort. 

d)  PSYCHOLOGICAL  STRESS  -  Task  completion 
causes  confusion,  frustration,  or  anxietv. 

Circle  No. 

Circle  No. 

1  =  No  demand 

1  =  None 

2  =  Little  demand 

2  =  Little 

3  =  Moderate  demand 

3  =  Moderate 

4  =  Heavy  demand 

4  =  High 

5  =  Very  heavy  demand 

5  =  Intense 

If  4  or  5,  indicate  why: 

If  4  or  5,  indicate  why: 

9.  Mission  successfully  performed  (even  with  problem):  YES  NO  (circle) 

10.  Fix  (procedure/task  modification  needed  to  complete  mission):  YES  NOT  (circle) 

11.  Recommended  fix  (if  any)  (continue  on  back  side,  if  necessary): _ 

On-Site  Data  Collector  PIN 

MANPRINT  Data  Collector  PIN 
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2.3.4  Use  of  Model  3  for  MANPRINT  Evaluation  of  ASAS  in  the  War  Fighting  Rapid 
Acquisition  Program  (WRAP) 

This  evaluation  involved  ten  different  types  of  ASAS  functions  with  no  well  defined  critical  task 
lists  that  could  be  compiled  within  the  short  response  time  available.  Whereas  some  tests  are 
designed  primarily  to  provide  opportunity  for  evaluation — with  any  training  function  being 
afforded  as  secondary  in  purpose — this  effort  was  designed  primarily  as  a  training  exercise  with 
an  evaluation  function  being  superimposed  (and  secondary)  on  the  behaviors  exhibited  during 
that  training. 

2.3.4. 1  Resources  Used  for  MANPRINT  Evaluation  of  ASAS  during  a  WRAP.  This  training 
exercise  was  conducted  as  a  three  day  activity  in  which  only  one  data  collection  resource  was 
available  to  provide  MANPRINT  Evaluation  for  soldiers  performing  in  the  ASAS  functionality. 
Contributing  to  the  uniqueness  of  the  methodology  employed  in  this  evaluation  was  its  use  to 
obtain  quality  MANPRINT  data  when  the  effort  required  a  quick  turn-around  activity.  Support 
for  this  assessment  began  on  Veteran's  Day  in  1997.  One  of  the  Fort  Hood  Field  Element 
personnel  was  tasked  to  coordinate  the  evaluation  effort  that  involved  several  MANPRINT 
Analysts,  but  generally  only  one  per  Battle  Field  Functional  Area  (BFA).  Some  of  these 
Analysts  came  from  U.S.  Army  Research  Laboratory  Headquarters  and  some  from  other  Field 
Elements.  The  author  was  contacted  at  about  10  a.m.  and  asked  if  he  could  support  this  effort  to 
collect  MANPRINT  data  for  ASAS.  With  about  three  hours  to  plan  how  this  support  could  be 
provided,  it  was  very  relevant  to  ask  what  kind  of  meaningful  and  potentially  useful 
MANPRINT  Evaluation  could  be  accomplished  with  so  little  time  for  preparation. 

2. 3. 4.2  Data  Collection  Procedures  for  MANPRINT  Evaluation  of  ASAS  during  a  WRAP.  An 
interview  protocol  containing  generic  MANPRINT  questions  (see  table  8)  was  the  basic  data 
collection  instrument  used  during  the  WRAP  for  ASAS.  With  the  soldier  sitting  alongside  the 
MANPRINT  Analyst,  each  MANPRINT  question  was  read  to  the  soldier,  first,  to  determine 
whether  there  was  any  likely  MANPRINT  problem.  For  those  questions  where  possible 
problems  did  exist,  the  Analyst  asked  for  description  of  the  problem(s)  and  continued  with 
follow-up  questions  until  a  relatively  complete  understanding  of  the  problem  existed.  Soldiers 
were  then  asked  for  their  recommendations  on  how  the  problem(s)  could  be  corrected  and  the 
consequences  likely  if  the  problem(s)  was  (were  not)  fixed.  Data  analysis  involved  tabulating 
reported  problems  by  MANPRINT  domain,  the  recommended  procedures  for  correction  and  the 
consequences  of  not  fixing  the  problem.  This  instrument  has  proven  to  be  a  useful  quick 
evaluation  instrument  on  several  occasions  for  collecting  MANPRINT  data  and  especially  useful 
when  you  don’t  know  very  much  about  the  system.  For  example,  when  table  8  is  used  to  support 
MANPRINT  Evaluation  for  the  SMART-T  in  the  FOTE,  generally  “SMART-T”  was  substituted 
for  ASAS  throughout. 
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Table  8.  MANPRINT  interview  questions  used  during  WRAP  for  ASAS. 


During  Your  Mission,  Did  You  ...  (Repeat  For  Each  Item  Below): 

N  Y 

MANPRINT  Domain 
Addressed 

1 .  See  any  task  or  operation  which  jeopardized  your  safety  or 
that  of  any  other  crewmember? 

N  Y 

Health  Hazard 

2.  Note  any  safety  problem  (actual  or  potential)  that  would 
degrade  (or  damage)  equipment,  configurations,  procedures  or 
work  conditions? 

N  Y 

System  Safety 

3.  Experience  or  suspect  any  condition  associated  with 

Operating  the  ASAS  which  you  consider  unhealthy? 

N  Y 

Health  Hazards 

4.  Or  any  other  crew  member  receive  any  injury  related  to  the 

ASAS  operations? 

N  Y 

Health  Hazard 

5.  Need  to  perform  any  task  or  operation  for  which  you  were 
not  trained? 

N  Y 

Training 

6.  Perform  any  task  or  operation  for  which  your  training  was  poor  or 
inadequate? 

N  Y 

Training 

7.  Note  any  crew/collective  task  degraded  because  instruction 
was  poor  or  tasks  taught  incompletely? 

N  Y 

Training 

8.  Feel  that  your  or  your  crew's  performance  was  degraded  or 
“held-down”  by  any  training  shortfall? 

N  Y 

Training 

9.  Perform  any  task  or  operation  that  in  your  judgment 
required  an  additional  crew  member  (or  more  time  than 
allowed  for  you  to  do  alone)? 

N  Y 

Manpower 

10.  Need  to  work  harder  with  ASAS  to  augment  MI  assessment  than 
with  manual  MI  procedures? 

N  Y 

Manpower 

11.  Perform  any  task  or  operation  that  was  outside  of  “normal”  duties  for 
your  MOS  and  skill  level? 

N  Y 

Personnel 

12.  Note  any  ASAS  equipment/hardware  that  was  unnecessarily  difficult, 
complicated,  or  “unfriendly”  to  work  with? 

N  Y 

Human  Factors 
Engineering:  Hardware 

13.  Experience  any  problem  associated  with  ASAS  software  at  your  crew 
station? 

N  Y 

Human  Factors 
Engineering:  Software 

14.  Identify  any  ASAS  equipment/hardware  placement  which  kept  you 
or  your  crew  from  “maxing  out”? 

N  Y 

Human  Factors 
Engineering:  Hardware 

15.  Note  any  ASAS  equipment/hardware  placement  which  made  access 
or  use  difficult? 

N  Y 

Human  Factors 
Engineering:  Hardware 

16.  Note  any  ASAS-related  procedures  which  were  unnecessarily 
difficult,  complicated,  or  “unfriendly”? 

N  Y 

Human  Factors 
Engineering:  Procedures 

1 7.  Identify  any  “dumb”  things  the  ASAS  required  you  to  do? 

N  Y 

Human  Factors 
Engineering:  Procedures 
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2.3.5  Use  of  Model  4  for  MANPRINT  Evaluation  of  the  Secure  Mobile  Anti- Jam  Reliable 
Tactical  Terminal  (SMART-T)  in  the  FOTE 

In  another  case,  it  was  necessary  for  a  second  MANPRINT  Analyst  to  replace  the  assigned 
MANPRINT  Analyst  after  the  Pilot  Test  for  a  Follow-on  Operational  Tests  and  Evaluation 
(FOTE)  of  the  SMART-T.  The  first  time  that  this  system  was  observed  by  the  second  Analyst 
was  following  the  Pilot  Test  of  the  FOTE  as  it  was  deployed  in  the  field. 

2.3.5. 1  Resources  Used  for  MANPRINT  Evaluation  of  the  SMART-T  in  the  FOTE.  In  this  test, 
there  were  about  30  soldiers  who  participated  as  operators;  system  operation  was  conducted  over 
a  three  day  period,  24  hr  per  day.  The  MANPRINT  Analyst  reviewed  and  modified  the  data 
collection  plans  and  instruments  used  planned  for  use  in  the  MANPRINT  evaluation;  ~20  of  the 
30  participating  soldiers  were  interviewed  with  the  instrument  shown  in  table  8  and  data  were 
analyzed  and  a  report  of  findings  was  prepared  for  the  Test  Officer  (75). 

2. 3. 5. 2  Data  Collection  Procedures  for  MANPRINT  Evaluation  of  the  SMART-T.  The 
interview  instrument  (see  table  8)  was  used  in  a  manner  very  similar  to  its  use  in  the  ASAS 
MANPRINT  Evaluation  during  the  WRAP  (see  paragraph  2. 3.4. 2).  Other  group-oriented 
MANPRINT  evaluation  instruments  were  administered  to  test  players  by  other  test  personal  and 
results  were  provided  to  the  MANPRINT  Analyst  for  analysis  and  summarization. 

2.3.6  Use  of  Model  4  for  MANPRINT  Evaluation  of  the  M1A2  in  the  IOTE 

Soldiers  participating  in  this  test  were  relatively  “low”  in  verbal  skills  and  a  usable  list  of  critical 
tasks  did  not  appear  available. 

2.3.6. 1  Resources  Used  for  MANPRINT  Evaluation  of  the  M1A2.  In  the  M1A2  IOTE,  there 
were  about  80  armor  MOSs.  Supporting  this  effort  were  a  GS13  (MGR)  who  developed  the  data 
collection  plan,  designed  the  data  collection  instruments  and  supervised  the  collection  and 
processing  of  data;  participating  in  the  data  collection  and  analysis  were  a  senior  retired  NCO 
with  system  experience  and  four  retired  NCOs  who  also  had  system  experience  served  as  Temps. 

2. 3. 6.2  Data  Collection  Procedures  for  MANPRINT  Evaluation  of  the  M1A2.  This  data 
collection  procedure  included  use  of  an  MANPRINT  Interview  Schedule  comparable  to  that  used 
in  Model  3  (table  8).  In  implementing  this  model,  there  was  an  attempt  to  validate  the  reported 
problem,  to  acquire  a  Risk  Assessment  for  Health  Hazards  and  System  Safety  (page  104)  and 
preparation  of  a  MANPRINT  Problem/Suggestion  Report  (page  105).  In  completion  of  this 
latter  fonn,  the  problems  reported  were  characterized  by  Type  (MANPRINT  domains)  and 
Mission  Impact;  data  collection  was  designed  to  describe  problems  and  included  opportunity  to 
acquire  suggestions  for  remediation. 
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SME/DATA  COLLECTOR  VALIDATION  INSTRUCTIONS 


Review  the  description  of  the  MANPRINT  problem. 

Attempt  to  validate  this  problem  though  one  of  the  following  means. 

•  Someone  else  who  may  have  knowledge  of  problem 

•  Interview  the  operator/RAM  data  collector/performance  data  collector  who  wrote  the  MPR 

•  Review  video  tape  of  problem 

After  you  have  located  one  of  the  above  sources  and  verified  the  problem,  ask  the  following 
questions  and  record  answers  on  the  MPR. 

1 .  What  were  the  contributing  causes  to  the  problem? 

2.  What  are  the  likely  consequences  if  the  problem  is  not  fixed? 

3.  What  are  the  solutions  to  this  problem? 

Remember  to  complete  the  MPR  by  doing  the  following: 

Mark  the  space  on  the  MPR  showing  how  you  validated  the  problem. 

Be  certain  to  fill  in  your  evaluation  of  the  Type  and  Impact  of  the  problem. 

Use  the  Safety  Hazard  and  Hazard  Probability  scales  to  determine  Safety  Category  and  Probability  codes 
at  the  bottom  of  the  MANPRINT  Problem/Suggestion  Report. 


SAFETY  HAZARD  CATEGORY 
Catastrophic  1 

Critical  II 

Marginal  III 

Negligible  IV 


Death  or  system  loss 

Severe  injury  or  major  system  damage 

Minor  injury  or  system  damage 

Less  than  minor  injury  or  system  damage 


HAZARD  PROBABILITY  CATEGORIES 

Description 

Level 

Specific  Individual  Item 

Inventory 

Frequent 

A 

Likely  to  occur  frequently 

Continuously  experienced 

Probable 

B 

Will  occur  several  times  in  life  of  an  item 

Will  occur  frequently 

Occasional 

C 

Likely  to  occur  sometime  in  life  of  an  item 

Will  occur  several  times 

Remote 

D 

Unlikely,  but  possible  to  occur  in  life  of  item 

Unlikely,  but  can  reasonably 
be  expected  to  occur 

Improbable 

E 

So  unlikely,  it  can  be  assumed  as  not 
occurring 

Unlikely  to  occur,  but  possible 
may  not  be  experienced 
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MANPRINT  Log  No. _ 

M1A2  BLOCK  II IOTE:  MANPRINT  PROBLEM/SUGGESTION  REPORT 

Operator:  _ Rank: _ Date:  _ 

Data  Collector:  _ PIN  No. _ 

Bumper  No.:  _ Vehicle  SN:  _ 

INSTRUCTIONS:  As  you  work  with  the  M1A2,  you  may  notice  or  experience  a  problem  or  have  a  suggestion  for 
improving  the  system.  For  any  problem  or  incident  you  encounter,  indicate  the  time  it  occurred  and  describe  it 
clearly.  Using  the  scales  below,  indicate  the  type  of  problem  and  its  impact  on  the  mission.  Simply  describe 
suggestions.  To  list  more  than  one  problem  or  suggestion,  draw  a  line  across  the  page  after  each  complete  entry. 


Type 

Mission  Impact 

E  =  Equipment  (hardware) 

C  =  Computer  software 

T  =  Training  (no/inadequate) 

S  =  Safety/Health  Hazard* 

W  =  Workload/Manpower 

O  =  Operator  (mistake/memory) 

S  =  Severe  (probable  failure) 

Mo  =  Moderate  (probably  degrading) 

Mi  =  Minimal  (mainly  attitudinal) 

N  =  Negligible  (essentially  none) 

TIME 

DESCRIPTION 

TYPE 

IMPACT 

SHORT  TITLE: 

SME  Badge  No. 

MPT  Team  O  None  O  Observed  O  Op  Interview  O  SME  O  Video 
VALIDATION  O  Invalid  O  Valid-^  Type  Code _  Impact  Code _ 


*  Safety  Category _  Probability 

DAG  Approval _  DTG _ 
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3.  Conclusions 


The  purpose  of  this  report  has  been  to  identify  alternative  procedures  for  conducting 
MANPRINT  evaluations.  After  characterization  of  the  areas  (domains)  addressed  by 
MANPRINT,  models  for  evaluation  were  discussed.  For  each  of  the  models,  procedures  for 
implementing  MANPRINT  evaluations  were  illustrated  by  presentation  of  different  data 
collection  instruments.  The  appendices  contain  additional  forms  and  some  detailed  procedures 
about  how  those  data  are  used — specifically  for  the  Personnel,  Health  Hazards,  and  System 
Safety  domains. 
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Appendix  A.  Memorandum  for  Defense  Manpower  Data  Center  (DMDC)  to 
Support  Sample  Representativeness  Comparisons 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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AMSRL-HR-MV  (70- ly) 


14  May  1999 


MEMORANDUM  FOR  DEFENSE  MANPOWER  DATA  CENTER  (DMDC),  DOD  CENTER, 

MONTEREY  BAY,  ATTN:  ,  400  GIGLING 

ROAD,  SEASIDE,  CA  93955-6771 


SUBJECT:  Request  for  ASVAB  and  Related  Data 


1.  Per  recent  FONECON  with  about  expected  SUBJECT  request,  paragraphs 

2-6  detail  the  current  request  being  made.  As  this  request  is  very  similar  to  a  recent  effort  completed  by  DMDC 
Analyst  (insert  name),  you  may  wish  to  assign  this  current  effort  to  him. 

2.  For  all  soldiers  on  the  enclosed  floppy1,  please  provide  individual  scores  and  group  mean  and  standard 
deviation  for  each  of  the  listed  variables  (except  as  noted): 


Date  of  Birth — YR 

(no  mean  or  standard  deviation) 

Date  of  Birth — MO 

(no  mean  or  standard  deviation) 

Date  of  Birth — DAY 

(no  mean  or  standard  deviation) 

Sex  (Gender) 

(no  mean  or  standard  deviation) 

Race  Ethnic 

(no  mean  or  standard  deviation) 

Paygrade 

(no  mean  or  standard  deviation) 

Education 

(variable  in  column  1 1  on  Active  Duty  Military  Master  and  Loss 
Edit  file) 

(no  mean  or  standard  deviation) 

Highest  Year  Education 

(variable  in  column  24  of  the  Active  Duty. .  .file) 

(no  mean  or  standard  deviation) 

Date  of  Entry — YR 

(no  mean  or  standard  deviation) 

Date  of  Entry — MO 

(no  mean  or  standard  deviation) 

Date  of  Entry — DAY 

(no  mean  or  standard  deviation) 

AFQT  Test  Group 
AFQT  Percentile 

(no  mean  or  standard  deviation) 

Standardized  Subtest  Scores 

GS 

AR 

WK 

PC 

NO 

CS 

AS 

MK 

MC 

El 

VE 

Army  ASVAB  Composites 

GT 

GM 

EL 

CL 

MM 

SC 

CO 

FA 

OF 

1 

ST 

Enclosed  floppy  contains  data  for  three  commissioned  officers.  It  is  understood  that  no  AFQT  and  ASVAB 
Scaled  Subtests  or  Composites  will  exist  for  those  soldiers. 
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3.  For  all  Active  duty  soldiers  (Army)  holding  MOS  96B,  please  provide  N  (population  size),  means  and  standard 
deviation  of: 

Age  (use  Date  of  Birth  on  file  and  1  Mar  99  as  current  date  in  computation) 

Time  in  Service  (use  Date  of  Entry  on  file  and  1  Mar  99  as  current  date  in  computation) 

AFQT  Percentile 

All  Standardized  Subtest  Scores  (referenced  in  para  2  above) 

All  Army  ASVAB  Composites  (referenced  in  para  2  above) 

4.  For  all  Active  duty  soldiers  (Army)  holding  MOS  96B,  please  provide  frequency  distributions  for: 

Paygrade 
Sex  (Gender) 

Race  Ethnic 
AFQT  Test  Group 

Education  with  categories'  definitions 

Flighest  Year  of  Education  with  categories'  definitions 

5.  For  all  Active  duty  commissioned  officers  holding  a  35D  MOS,  please  provide,  by  rank  (paygrade),  N 
(population  size)  and  frequency  distributions  for: 

Sex  (Gender) 

Race  Ethnic 

Education  (variable  in  column  1 1  on  the  Active  Duty. .  .file) 

Flighest  Year  of  Education  (variable  in  column  24  of  the  Active  Duty...  file) 

Paygrade 

6.  For  all  Active  duty  commissioned  officers  holding  a  35D  MOS,  please  provide,  by  rank  (paygrade),  N 
(population  size)  and  mean  and  standard  deviations  of 

Age  (use  Date  of  Birth  and  1  Mar  99  as  current  date  in  computation) 

Time  in  Service  (use  Date  of  Entry  on  file  and  1  Mar  99  as  current  date  in  computation) 

7.  File  on  floppy  has  been  created  with  MS  Word  and  saved  in  two  forms:  a)  Text  and  b)  Rich  Text  format. 
Please  direct  any  questions  concerning  this  request  to  Dr.  Otto  Fleuckeroth,  DSN  738-9377/9572,  commercial  (254) 
288-9377/9572. 

8.  Please  forward  all  output  to: 

Army  Research  Laboratory 

ATTN:  AMSRL-HR-MV  (Dr.  Heuckeroth) 

HQ  TEXCOM 

91012  Station  Ave 

Fort  Hood,  TX  76544-5073 

Fax,  if  convenient,  to  DSN  738-1691,  commercial  (254)  288-1691. 


OTTO  H.  HEUCKEROTH 
Research  Psychologist 

Enel 

Floppy  disk  (flat  file) 

95-OT-l  140A 
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Intentionally  left  blank. 
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Appendix  B.  Sample  Representativeness  of  Soldiers  Participating  in  the 

March  1999  DT/OT  RWS  LUTE 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Sample  Representativeness  of  Soldiers  Participating  in  the  March  1999  DT/OT  RWS  LUTE 


INTRODUCTION 

Between  16  and  19  March  1999,  enlisted  soldiers  holding  an  MOS  96B  from  the  3rd  BDE  of  the  1st  CD 
and  B  Co.  of  the  104th  MI  BN  participated  in  a  Tester  controlled  DT/OT  event.  In  addition  to  these  10  enlisted 
soldiers,  these  three  commissioned  officers— a  MAJ  and  a  1LT  from  the  3rd  BDE  of  the  1st  Co.  can  one  2LT  from  B 
Co.  of  the  104th  MI  BN. 


METHOD 

In  addition  to  the  14  instruments  used  by  ARL  to  facilitate  the  MANPRINT  assessment— from  individual 
RWS  task  classroom  training  assessment  through  the  MANPRINT  assessment  during  the  Record  test— data  on 
selected  demographic  and  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  data  were  obtained  for  enlisted 
personnel;  only  selected  demographic  data  were  obtained  for  the  participating  commissioned  officers.3  Variables 
used  to  make  sample  representativeness  comparisons  for  the  enlisted  soldiers  include:  1)  Each  of  the  1 1  ASVAB 
Standardized  Subtest  Scores;  2)  Armed  Force  Qualification  Test  (AFQT)  percentile;  3)  Each  of  the  10  ASVAB 
Composites;  4  Age;  5)  Time  in  Service;  6)  Gender;  7)  Race/Ethnicity  and  8)  Education.  Variables  used  to  make 
sample  representativeness  judgments  about  participating  commissioned  officers  include:  1)  Gender;  2) 
Race/Ethnicity;  3)  Education;  4)  Time  in  Service;  and  5)  Age.  Coordination  with  DMDC  indicated  that  Composite 
scores  in  the  Active  Duty  File  were  suspect  in  part,  due  to  file  formatting  problems.  In  some  cases,  some  Subtest 
Scores,  from  which  Composites  were  computed,  were  missing;  these  records  were  deleted  in  the  descriptive 
statistics  presented  for  comparisons  involving  Composites.  Finally,  Composite  data  for  Active  Duty  soldiers  who 
entered  the  service  prior  to  1981  were  not  normalized  such  that  they  are  comparable  to  ASVAB  Composites  data 
computed  after  1980.  For  every  Composite  of  about  500  Active  Duty  soldiers  in  the  pre  '81  aggregate  population 
(MOS  96B,  96D,  98C),  the  mean  normalized  Composites  are  smaller  and  variances  larger  than  those  for  comparable 
Active  Duty  soldiers  taking  the  ASVAB  after  1980.  To  reduce  ambiguity  in  interpretation,  only  the  Composites  for 
that  portion  of  the  aggregate  Active  Duty  population  who  took  the  ASVAB  after  1980  are  reported  in  Table  1. 


RESULTS  AND  DISCUSSION 

Sample  Representativeness  Comparisons  for  Enlisted  MOS  96B  Soldiers  Participating  in  the  RWS  Enclave  During 
the  March  1999  DT/OT  LUTE. 

Table  1  provides  the  means  and  standard  deviations  of  ASVAB  Scaled  Subtests, 

Composites  and  Armed  Forces  Qualification  Test  (AFQT)  percentile  for  active  duty  soldiers  holding  MOS  96B; 
comparable  data  are  provided  for  the  10  MOS  96B  who  participated  as  RWS  Analysts  during  the  DT/OT  test  in 
March  1999.  Also  included  in  this  table  are  statistics  and  parameters  for  soldier  time  in  service  and  age. 


3 

The  source  of  both  ASVAB  and  the  selected  demographic  characteristics  was  from  the  Defense  Manpower  Data  Center 
(DMDC)  MEPCOM  and  Active  Duty  files. 
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Table  1 


Comparison  of  Mean  and  Variance  of  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  Scaled  Subtests  and 
Composites,  and  Age  and  Time  In  Service  of  the  Population  of  Active  Duty  Enlisted  Soldiers  Holding  MOS  96B 
and  Analysts  Operating  Remote  Work  Stations  (RWS)  During  the  March  1999  DT/OT  LUTE 


POPULATION  OF  ACTIVE  DUTY 
SOLDIERS  HOLDING 

MOS  96B,  96D  OR  98C 

PARTICIPANTS  IN  THE 
DECEMBER  1998  WFX 
SERVING  AS  RWS  ANALYSTS 

MEANS 

SIGNIFI¬ 

CANCE 

TESTING 

VARIANC 

E  SIGNIFI¬ 
CANCE 
TESTING 

Variable 

N 

Mean 

Std  Dev 

N 

Mean 

Std  Dev 

t  Test 

F 

AFQTP 

2406 

72.22 

15.78 

10 

80.50 

13.99 

1.66 

1.27 

GS 

2991 

56.43 

6.55 

10 

58.60 

8.33 

0.82 

1.62 

AR 

2991 

56.08 

6.77 

10 

59.40 

5.93 

1.77 

1.30 

WK 

2991 

55.69 

5.34 

10 

56.30 

4.92 

0.39 

1.18 

PC 

2991 

55.56 

5.10 

10 

57.10 

6.23 

0.78 

1.49 

NO 

2991 

55.35 

6.47 

10 

54.30 

5.72 

0.58 

1.28 

CS 

2991 

54.52 

7.56 

10 

57.30 

6.11 

1.44 

1.53 

AS 

2991 

51.80 

7.95 

10 

49.50 

9.58 

0.76 

1.45 

MK 

2991 

57.31 

7.15 

10 

62.90 

3.21 

5.46*** 

4  96*** 

MC 

2991 

55.87 

7.46 

10 

58.70 

3.68 

2.41* 

4  ii** 

El 

2991 

53.38 

7.42 

10 

50.50 

10.01 

0.91 

1.82 

VE 

2991 

55.38 

5.64 

10 

57.00 

5.21 

0.98 

1.17 

GT 

2400 

113.55 

9.54 

10 

117.90 

10.09 

1.44 

1.12 

GM 

2400 

112.04 

11.18 

10 

112.50 

15.25 

0.13 

1.86 

EL 

2406 

114.10 

10.54 

10 

117.80 

12.02 

1.11 

1.30 

CL 

2400 

114.93 

9.78 

10 

121.30 

7.86 

2.06* 

1.55 

MM 

2400 

110.70 

11.60 

10 

108.00 

10.82 

0.73 

1.15 

SC 

2400 

112.06 

10.48 

10 

114.40 

10.72 

0.70 

1.05 

CO 

2400 

112.26 

11.69 

10 

115.80 

9.15 

0.96 

1.63 

FA 

2400 

115.46 

11.11 

10 

123.30 

7.21 

3  42*** 

2.37 

OF 

2400 

112.55 

9.69 

10 

112.10 

8.08 

0.15 

1.44 

ST 

2400 

116.01 

9.52 

10 

121.40 

8.26 

1.79 

1.33 

TIME  IN 
SERVICE 

2406 

77.40 

58.44 

10 

45.00 

50.40 

1.75 

1.34 

AGE  (yrs) 

2406 

27.13 

5.99 

10 

25.57 

6.58 

0.82 

1.21 

***  p  <  oi 
**  P  <  .025 
*  P  <  .05 


Tests  for  equality  of  variances  were  compared  with  F  tests;  comparisons  of  means  were  assessed  by  use  of  t 
tests.  Comparison  of  soldier  participants  in  the  WFX  LUTE  with  the  population  distributions  of  gender,  race  and 
education  were  assessed  by  comparison  of  observed  and  expected  frequencies  (see  Tables  2  -  5).  As  the  expected 
frequencies  in  at  least  one  category  were  less  than  5,  j2  goodness-of-fit  tests  would  be  of  questionable  validity. 
“Inter-ocular”  comparisons  of  observed  and  expected  frequencies  were  performed. 
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Table  2 


Observed  and  Expected  Frequency  of  Enlisted  Soldiers  Participating  as  RWS  Analysts  During  the  March  1999 
DT/OT  LUTE  by  Categories  of  Gender 


Number  of 

Active  Duty  Soldiers 
(MOS  96B) 

Expected 

Proportions 

Observed  and  Expected  Frequency  of  RWS 
Test  Participants 

Category  of  Gender 

Observed 

Expected 

1932 

.805 

7 

8.05 

MALE 

468 

195 

3 

1.95 

FEMALE 

2400 

10 

TOTAL 

Table  3 

Observed  and  Expected  Frequency  of  Soldiers  Participating  as  RWS  Analysts  During  the  March  1999  DT/OIT 
LUTE  by  Categories  of  Race 


Number  of  Active 
Duty  Soldiers 
(MOS  96B) 

Expected 

Proportions 

Observed  and  Expected  Frequency  of  RWS 
Test  Participants 

Category  of  Race 

Observed 

Expected 

1776 

.740 

8 

7.4 

WHITE 

379 

.158 

1 

1.58 

BLACK 

112 

.047 

1 

.47 

HISPANIC 

133 

.055 

0 

.55 

OTHER 

2400 

10 

TOTAL 
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Table  4 


Observed  and  Expected  Frequency  of  Enlisted  Soldiers  participating  as  RWS  Analysts  During  the  March  1999 
DT/OT  LUTE  by  Categories  of  Education 


Number  of  Active 
Duty  Soldiers 
(MOS  96B) 

Expected 

Proportions 

Observed  and  Expected  Frequency  of  RWS 
Test  Participants 

Category  of 

Education 

Observed 

Expected 

2047 

.853 

6 

8.53 

HS  Diploma  or  GED 

79 

.033 

1 

.33 

2  Yrs  College 

59 

.025 

0 

.25 

3-4  Yrs  College,  No 
Diploma 

183 

.076 

3 

.76 

College  Degree  (4 

Yrs) 

14 

.006 

0 

.06 

Masters  Degree 

1 

.000 

0 

.00 

Doctorate 

17 

.007 

0 

.07 

Unknown 

2400 

10 

Total 

Sample  Representativeness  of  MOS  35D  Commissioned  Officers  Participating  in  the  RWS  Enclave  During  the 
March  1999  DT/OT  LUTE. 

As  the  RWS  enclave  contained  only  three  35D  MOS  officers— a  male  MAJ  as  S2,  a  female  LT1  as  AS2  and 
a  female  LT2  as  ACT  chief— statistical  comparisons  were  not  feasible.  To  make  an  assessment  of  sample 
representativeness  for  officers  with  such  small  sample  sizes,  comparisons  on  gender,  race  and  education  variables 
are  made  by  comparing  the  model  frequency  of  active  duty  commissioned  officers— for  the  ranks  participating  in  the 
test— with  the  category  value  of  the  test  participant  (see  Tables  5-8).  It  is  assumed  that  commissioned  officers  of 
each  rank  used  in  the  test  were  assigned  to  enclave  functionality  positions  consistent  with  established  MI  doctrine. 
For  time  in  service  and  age  variables,  means  and  standard  deviations  for  each  participating  Active  Duty  Rank  is 
shown  together  with  the  relevant  information  for  each  rank  participating  in  the  March  1999  RWS  DT/OT  LUTE  (see 
Table  8). 
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Table  5 


Frequency  of  Active  Duty  Commissioned  Officers  Holding  MOS  35D  in  March  1999  by  Gender1 


Number  of  Active  Duty  Officers  by 
Category 

Rank  of  Commissioned  Officers  Participating  in  the  TEST 

2LT 

1LT 

MAJ 

Males 

193 

533 

634* 

Females 

105* 

180* 

86 

Asterisks  indicate  the  Gender  of  each  MOS  35D  test  participant. 


Table  6 


Frequency  of  Active  Duty  Commissioned  Officers  Holding  MOS  35D  in  March  1999  by  Race/Ethnicity1 


Number  of  Active  Duty  Officers  by 
Category 

Rank  of  Commissioned  Officers  Participating  in  the  TEST 

2LT 

1LT 

MAJ 

White 

219* 

548* 

580* 

Black 

40 

81 

67 

Hispanic 

19 

30 

44 

Other 

20 

47 

23 

'Asterisks  indicated  the  Race  of  each  MOS  35D  test  participant. 
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Table  7 


Frequency  of  Active  Duty  Commissioned  Officers  Holding  MOS  35D  in  March  1999  by  Education1 


Number  of  Active  Duty  Officers  by 
Category 

Rank  of  Commissioned  Officers  Participating  in  the  TEST 

2LT 

1LT 

MAJ 

HS/GED 

3 

3 

0 

2  Yrs  College 

3 

9 

1 

BS/BA 

255* 

605* 

372* 

MS/M  A/ 1st  Profn 

6 

25 

337 

Doctorate 

0 

0 

1 

Asterisks  indicated  the  level  of  education  attained  by  the  Commissioned  Officer  at  the  indicated  rank  who 
participated  in  the  March  1999  DT/OT  LUTE. 


Table  8 


Summary  Descriptive  Statistics  for  Age  and  Time  in  Service  for  2LTs,  lLTs  and  MAJs  Holding  MOS  35D1 


2LT 

1LT 

MAJ 

AGE 

N 

298 

710 

719 

Mean 

25.98  (22.75) 

27.11  (23.83) 

38.08  (39.17) 

Std  Dev. 

2.62 

2.84 

3.18 

Time  in  Service 

N 

298 

713 

720 

Mean 

2.46  (0.75) 

4.31  (1.83) 

14.80  (13.42) 

Std  Dev. 

2.58 

2.67 

2.56 

'Number  in  parentheses  is  the  Age  and  Time  in  Service  of  Commissioned  Officers  who  participated  in  the  march 


1999  DT/OT  LUTE. 
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CONCLUSIONS 


Enlisted  Soldier  Participants 

Compared  to  the  population  of  Active  Duty  MOS  96B  soldiers,  test  participants  were 
significantly  more  homogeneous  and  had  significantly  higher  aptitude  scores  for  Mathematics 
Knowledge  (MK)  and  Mechanical  Comprehension  (MC)  Subtests  compared  to  the  population  of 
Active  Duty  MOS  96B  (see  Table  1).  Both  of  these  Subtest  raw  scores  are  used  to  compute 
ASVAB  Composites  FA  and  ST.  While  the  enlisted  test  participants  also  show  the  same 
significance  pattern  for  the  FA  Composite,  differences  for  the  ST  Composite— a  Composite  for 
which  entry  into  MOS  96B  is  set  at  105  as  a  minimum  (AR  61 1-201)— while  in  the  same 
direction,  are  not  significant.  In  addition,  mean  CL  aptitude  is  significantly  greater  for  MOS  96B 
for  test  participants  than  for  the  population  from  which  they  were  selected.  These  differences 
might,  in  part,  be  attributed  to  the  fact  that  enlisted  MOS  96B  test  participants  were  slightly  more 
educated  (see  Table  4)  than  the  population  from  which  they  were  selected.  No  marked 
differences  in  gender  (Table  2)  or  Race  (Table  3)  distributions  were  noted.  These  findings 
together  with  the  observation  that  the  differences  for  many  of  the  other  Subtests  that  the  enlisted 
MOS  96B  test  participants  are  slightly  superior  in  aptitude  and  education  compared  to  target 
audience  population  from  which  they  were  selected.  Had  these  differences  not  existed,  it  is  this 
analyst's  judgment  that  the  training  (need  to  integrate  conceptual  MOS  TTPs  training  with 
training  on  use  of  automated  software)  and  software  problems  exhibited  would  have  been  even 
more  severe.  With  modification  in  the  training  (as  recommended)  and  correction  of  software 
problems,  it  is  this  analyst's  judgment  that  with  a  more  representative  set  of  enlisted  participants 
the  automated  RWS  function  would  be  demonstrated  as  effective  and  suitable,  i.e.,  the  slight  lack 
of  sample  representativeness  is  much  less  important  to  the  overall  system  evaluation  than  the 
training  deficiencies  and  HFE  (software)  problems  evidenced. 

Commissioned  Officers  Participants 

Apart  from  the  fact  that  the  Assistant  S2  (AS2)  and  ACT  Chief  were  younger  and  with 
less  experience  than  the  population  of  MOS  35D  commissioned  officers  from  which  they  were 
selected,  no  concern  about  representativeness  is  evident.  Both  AS2  and  the  ACT  Chief  appeared 
quite  competent  and  seemed  to  work  well  with  the  S2  and  enlisted  analysts.  As  with  the  enlisted 
analysts  test  participants,  any  slight  unrepresentativeness  noted  is  much  less  important  to  the 
evaluation  assessment  than  addressing  training  deficiencies  and  correcting  software  problems 
evidenced. 
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Appendix  C.  Human  Factors  Engineering  Questionnaire  for  an  Electronic 

Equipment  Test  Facility  (EETF) 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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HUMAN  FACTORS  QUESTIONNAIRE 


ELECTRONIC  TEST  FACILITY  OPERATOR  (35C20) 


NAME 


DATE 


The  purpose  of  the  following  questionnaire  is  to  obtain  your  opinions 
and  observations  about  the  adequacy  of  the  AN/MSM  105  Electronic  Test 
Facility  from  an  operator's  point  of  view.  Take  as  much  time  as  you  feel 
is  necessary  to  accurately  complete  the  questionnaire.  The  administrator 
will  answer  any  questions  you  have. 
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1.  ENVIRONMENT 


Using  the  scale  to  the  right 

<D 

indicate  with  a  check  mark  (V) 

3 

<D 

c3 

to  indicate  the  adequacy  of  the 

td 

3 

§■ 

following  environmental 

0) 

CD 

T3 

conditions  in  the  ETF: 

13 

C 

£3 

13 

Th 

o 

U 

C/3 

O 

5 

in 

o 

CQ 

C/3 

O 

2 

1.  Temperature 

2.  Ventilation 

3.  Noise  _  _  _ 

4.  Vibration  _  _  _ 

5.  Illumination  _  _  _ 

B.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 


a 

§■ 

<D 

03 

gj 

13 


o 

U 
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II.  EQUIPMENT  CHARACTERISTICS 

A.  VIDEO  DISPLAY  UNIT 

Using  the  scale  to  the  right 
indicate  with  a  check  mark  (a/) 
to  indicate  the  adequacy  of  the 
following  environmental 
conditions  in  the  ETF : 

1.  DISPLAY 

a.  Display  brightness 

b.  Absence  of  glare 

c.  Absence  of  flicker 

d.  Letter  discrimination 

e.  Viewing  distance 

f.  Angle  of  view 

g.  Location  of  display 

h.  Other  (specify) _ 

2.  INDICATOR  PANEL 

a.  Display  brightness 

b.  Absence  of  glare 

c.  Absence  of  flicker 

d.  Viewing  distance 

e.  Angle  of  view 

f.  Correct  labels 

g.  Location  of  indicators 

h.  Other  (specify) _ 


<D 


a 

3 

CD 

a* 

CD 

CD 

03 

'o3 

=3 

c 

§- 

cr 

CD 

CD 

*0 

CD 

13 

t3 

c 

£ 

Th 

CD 

T3 

S-H 

O 

CQ 

o 

U 

c n 

O 

5 

t/5 

O 

5 
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Completely  Inadequate 


3.  KEYBOARD  AND  CONTROLS 


<D 


03 

2 

CD 

<D 

<D 

a 

— 1 

C 

§< 

cr 

CD 

>% 

<D 

<D 

13 

c 

<D 

^l~> 

Oh 

CD 

a 

o 

U 

C/3 

o 

5 

Jh 

O 

CQ 

C/3 

o 

2 

a.  Size 

b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 

e.  Correct/Complete  labels 

f.  Understandable  labels 

g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 
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Completely  Inadequate 


0) 


<D 

03 

d 

d 

<L> 

cr 

a" 

0) 

<D 

0) 

*d 

ts 

c 

& 

dH 

<D 

0) 

-d 

0) 

•d 

<D 

CD 

c 

a 

CD 

<D 

^l~> 

Th 

Dh 

0) 

Ph 

a 

o 

U 

C/2 

O 

s 

’d 

S-H 

o 

CQ 

C/3 

o 

2 

a 

o 

q.  Control  type  (type  of 
control  is  appropriate 
for  type  of  function) 

r.  Other  (specify) _ 


B.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 
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B.  LINE  PRINTER 


Using  the  scale  to  the  light 
indicate  with  a  check  mark  (a/) 
to  indicate  the  adequacy  of  the 
following  environmental 
conditions  in  the  ETF : 


1.  INDICATOR  LIGHTS 

a.  Brightness 

b.  Absence  of  glare 

c.  Absence  of  flicker 

d.  Viewing  distance 

e.  Angle  of  view 

f.  Correct  labels 

g.  Location  of  indicators 

h.  Indicators  inform  you  of  what 
you  need  to  know 

(1)  in  a  timely  manner 

(2)  with  enough  precision 

(3)  with  relevant  information 

i.  Other  (specify) _ 


CD 

ccS 

O 

CD 

O* 

0) 

CD 

ts 

ccS 

O 

c 

§• 

cr 

CD 

CD 

’T3 

CD 

ccS 

13 

< 

.O 

CD 

Oh 

CD 

a 

C/3 

S-H 

C/3 

(*■*1 

o 

U 

CD 

2 

O 

CQ 

CD 

2 

2.  CONTROLS 

a.  Size 

b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 
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Completely  Inadequate 


e.  Correct/Complete  labels 


a 

§• 

<D 

’Td 

c 


Oh 

a 

o 

U 


D 


0) 

cS 

"cd 

d 

d 

a" 

CT" 

0) 

0) 

'Td 

-o 

<D 

aJ 

c 

.Cl 

Th 

’Td 

C/3 

C/3 

o 

J-H 

Q 

o 

5 

CQ 

S 

f.  Understandable  labels 

g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 

q.  Control  type  (type  of  control 
is  appropriate  for  type  of 
function) 

r.  Other  (Specify) _ 
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Completely  Inadequate 


B.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 
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C.  CONTROL  STATION 


Using  the  scale  to  the  right 
indicate  with  a  check  mark  (a/) 
to  indicate  the  adequacy  of  the 
following  environmental 
conditions  in  the  ETF : 


a 

0) 

"O 


I 

o 

U 


<D 

a 

3 

0) 

0) 

< 

.g 

j>> 

Th 

0) 

C/3 

-o 

o 

S 

S-H 

O 

CQ 

C3 

§■ 

u 

CS 

e 


1.  COUNTERS  AND  INDICATORS 

a.  Brightness 

b.  Absence  of  glare 

c.  Absence  of  flicker 

d.  Viewing  distance 

e.  Angle  of  view 

f.  Correct  labels 

g.  Location  of  indicators 

h.  Counters  and  Indicators  inform 
you  of  what  you  need  to  know 

(1)  in  a  timely  manner 

(2)  with  enough  precision 

(3)  with  relevant  information 

i.  Other 

(specify) _ 

2.  CONTROLS 

a.  Size 

b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 
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Completely  Inadequate 


0) 


a 

0) 

<D 

a" 

E 

<D 

a* 

0) 

(L> 

< 

0) 

’id 

Th 

os 

< 

<D 

£ 

E 

El 

C/3 
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e.  Correct/Complete  labels 

f.  Understandable  labels 

g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 

q.  Control  type  (type  of  control 
is  appropriate  for  type  of 
function) 

r.  Other  (Specify) _ 
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Completely  Inadequate 


3.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 
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D.  DC  POWER  STATION 


Using  the  scale  to  the  light 
indicate  with  a  check  mark  (a/) 
to  indicate  the  adequacy  of  the 
following  environmental 
conditions  in  the  ETF: 


1.  METERS  AND  INDICATORS 

a.  Brightness 

b.  Absence  of  glare 

c.  Absence  of  flicker 

d.  Viewing  distance 

e.  Angle  of  view 

f.  Correct  labels 

g.  Location  of  meters  and 
indicators 

h.  Meters  and  Indicators  inform 
you  of  what  you  need  to  know 

(1)  in  a  timely  manner 

(2)  with  enough  precision 

(3)  with  relevant  information 

i.  Other 

(specify) _ 

2.  CONTROLS 

a.  Size 

b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 

e.  Correct/Complete  labels 
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Completely  Inadequate 


f.  Understandable  labels 


g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 

q.  Control  type  (type  of  control 
is  appropriate  for  type  of 
function) 

r.  Other  (Specify) _ 
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Completely  Inadequate 


3.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 
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E.  TAPE  STATION 


Using  the  scale  to  the  right 
indicate  with  a  check  mark  (a/) 
to  indicate  the  adequacy  of  the 
following  environmental 
conditions  in  the  ETF : 
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1.  INDICATOR  LIGHTS 

a.  Brightness 

b.  Absence  of  glare 

c.  Absence  of  flicker 

d.  Viewing  distance 

e.  Angle  of  view 

f.  Correct  labels 

g.  Location  of  indicators 

h.  Indicators  inform  you  of  what 
you  need  to  know 

(1)  in  a  timely  manner 

(2)  with  enough  precision 

(3)  with  relevant  information 

i.  Other 

(specify) _ 

2.  CONTROLS 

a.  Size 

b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 
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Completely  Inadequate 


e.  Correct/Complete  labels 
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f.  Understandable  labels 

g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 

q.  Control  type  (type  of  control 
is  appropriate  for  type  of 
function) 

r.  Other  (Specify) _ 
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Completely  Inadequate 


3.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 
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F.  UNIT  UNDER  TEST  (UUT)  STATION 


Using  the  scale  to  the  right  indicate  with 
a  check  mark  (V)  how  adequate  the  UUT 
Station  is  in  each  of  the  following 
areas: 


1.  CONTROLS 

a.  Size 

b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 

e.  Correct/Complete  labels 

f.  Understandable  labels 

g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 
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Completely  Inadequate 


ID 

CD 

td 

2 

ID 

a" 

a" 

(D 

CD 

ID 

P , 

c3 

c 

& 

O" 

ID 

<D 

CD 

a 

CD 

c 

.Cl 

CD 

Th 

^l~> 

a, 

<D 

Ph 

a 

o 

U 

C/3 

o 

2 

S-H 

O 

CQ 

C/3 

o 

5 

a 

o 

U 

q.  Control  type  (type  of 
control  is  appropriate 
for  type  of  function) 

r.  Other  (specify) _ 


B.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 
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G.  PROGRAMMABLE  INTERFACE  UNIT 

(PIU)  STATION 

0) 

Using  the  scale  to  the  right  indicate 

& 

ID 

ID 

with  a  check  mark  (V)  how  adequate 
the  PIU)  Station  is  in  each  of  the 
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following  areas: 
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a.  Size 


b.  Shape 

c.  Spacing  between  controls 

d.  Resistance  (too  easy  to 
turn  or  push,  or  too  hard 
to  turn  or  push) 

e.  Correct/Complete  labels 

f.  Understandable  labels 

g.  Size  of  labels 

h.  Location  of  labels 

i.  Absence  of  unrelated  or 
confusing  markings 

j.  Visibility  of  controls 

k.  Angle  of  view 

l.  Location  of  critical  controls 

m.  Reach  distance  of 
critical  controls 

n.  Location  of  non-critical 
controls 

o.  Reach  distance  of 
non-critical  controls 

p.  Functional  grouping  (controls 
with  related  functions  are 
grouped  together) 
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Completely  Inadequate 
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q.  Control  type  (type  of 
control  is  appropriate 
for  type  of  function) 

r.  Other 

(specify) _ 


2.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate  responses 


142 


III.  OVERALL  CONFIGURATION  OF  ETF 
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A.  EQUIPMENT  LOCATION 
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td 

0) 

Using  the  scale  to  the  right 
indicate  with  a  check  mark  (a/) 
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the  adequacy  of  the  location 
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within  the  ETF  of  each  of  the 
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following  pieces  of  equipment: 
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1.  Video  Display  Terminal 

_ 

_ 

_ 

_ 

_ 

2.  Printer 

_ 

_ 

_ 

_ 

_ 

3.  UUT  Station 

_ 

_ 

_ 

_ 

_ 

4.  PIU  Station 

_ 

_ 

_ 

_ 

_ 

5.  DC  Power  Station 

_ 

_ 

_ 

_ 

_ 

6.  Control  Station 

_ 

_ 

_ 

_ 

_ 

7.  Tape  Station 

— 

— 

— 

— 

— 

8.  Storage  Racks 

— 

— 

— 

— 

— 

9.  Storage  Cabinets 

— 

— 

— 

— 

— 

10.  Work  Bench 

— 

— 

— 

— 

— 

11.  Work  Desk 

_ 

_ 

_ 

_ 

_ 

12.  Cables 

_ 

_ 

_ 

_ 

_ 

13.  Other  (specify) 

14.  Explanation  of  Borderline,  Mostly  Inadequate,  and  Completely  Inadequate  responses. 
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B.  WORKSPACE 


Using  the  scale  to  the  right 
indicate  with  a  check  mark  V) 
the  adequacy  of  the  Workspace 
and  Storage  Space  within  the  ETF 


1 .  Amount  of  workspace 

2.  Amount  of  storage  space 
for  tools 

3.  Amount  of  storage  space 
for  manuals 

4.  Amount  of  storage  space 
for  personal  gear 

5.  Amount  of  storage  space 
for  other  items 


6.  Explanation  of  Borderline,  Mostly  Inadequate,  and  Completely  Inadequate  responses. 
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Completely  Inadequate 


IV.  JOB  PROCEDURES 


Using  the  scale  to  the  right 
indicate  with  a  check  mark  (a/) 
how  easy  of  difficult  it  is  to 
perform  each  of  the  following 
procedures: 


A.  POWER-UP  THE  EQUIPMENT 

1 .  Prepare  the  ETF  for  operation 

2.  Cable  the  system  for  operation 

3.  Power  up  to  standby  power-on 

4.  Power  up  to  control  and 
display  subsystem  power-on 

5.  Power  up  to  frill  power-on 

6.  Recover  from  loss  of  power 

7.  Other  procedures  (specify) 


8.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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B.  MAGNETIC  TAPE  AND  DISK 
CARTRIDGE  HANDLING 
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1 .  Load  the  magnetic  tape 

2.  Unload  the  magnetic  tape 

3.  Install  the  disk  cartridge 

4.  Remove  the  disk  cartridge 

I 

5.  PREPARE  A  NEW  DISK  FOR  USE 

a.  Format  the  disk 

b.  Initialize  the  disk 

c.  Install  a  system  bootstrap 
loader 

d.  Transfer  OS  software 
from  tape  to  disk 

e.  Build  a  duplicate  disk 

6.  Prepare  an  OS  Diagnostic 
Disk  (DDOS) 

7.  Verify  a  disk 

8.  Generate  a  system  tape  using 
The  OS 

9.  Other  procedures  (specify) 


8.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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C.  COMPUTER  START-UP 
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1.  Normal  start  up  from  disk 

2.  Start  up  from  duplicate  disk 

3.  Start  up  from  magnetic  tape  _ 

4.  Restart  system  _ 

I 

5.  Other  procedures  (specify) 

6.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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D.  DATA  HANDLING 

1 .  System  release 

2.  Tape  release 

3.  Load  UUT  program  from 
tape  to  disk 

4.  Duplicate  disk  fde 

I 

5.  Delete  disk  file 

6.  Search  file  name 

7.  Display  file  content  on  VDT 

8.  Print  file  content  on  printer 

9.  Other  procedures  (specify) 
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10.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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E.  TESTING  THE  UUT 
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1.  Test  line  replaceable  units 
(LRUs)  using  the  410 

2.  Identify  the  UUT  to  determine 
the  hardware  and  software 
necessary  to  test  it 

3.  Select  and  install  the 
correct  memory  pack 

4.  Determine  the  required  mode 
of  equipment  operation 

5.  Connect  the  UUT  to  the  410 
while  following  the  display 
diagrams  on  the  VDT 

6.  Follow  the  TPS  program 
instructions 

7.  Probe  and  test  the  UUT 
while  following  the 
computer  instructions 

8.  Test  UUTs  with  465  DCT 

9.  Other  procedures  (specify) 


10.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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F.  SELF  TESTING  AND  ALIGNING 
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1.  Control  and  display  subsystem 
Testing 

a.  Perform  the  DDCS  self  test 

b.  Perform  the  DTOS  self  test 

c.  Perform  the  video  terminal 
self  test 

d.  Perform  the  line  printer  self 
test 

2.  Self  Testing 

a.  Perform  the  full  ILSST.IC  self 
test 

b.  Perform  the  UUT  Test  Survey 
Leader  self  test 

3.  Systems  Alignment  Run  the 
SYSCAL  Program 

a.  Manual  Mode 

b.  Automatic  Mode 

c.  Run  all  mode 

d.  Information  Mode 

4.  Perform  the  465  DCT  self  test 

5.  Other  procedures  (specify) 
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Did  Not  Perform 


6.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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G.  OPERATING  THE  TEST 
EQUIPMENT 

1.  Operate  the  computer  using  the 
Command  line  printer  (CLI) 

2.  Store  and  maintain  Test  Program 
Sets  (TPSs) 

3.  Load  paper  in  the  line  printer 

4.  Use  the  oscilloscope 

5.  Other  procedures  (specify) 
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6.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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H.  POWER  DOWN 
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1.  From  full  power-on  to  control  and 
display  subsystem  power-on 

2.  From  control  and  display  sub¬ 
system  power-on  to  standby 
power-on 

3.  From  standby  power-on  to  full 
power-off 

4.  From  any  power-on  state  to 
emergency  power-off 

5.  Other  procedures  (specify) 


6.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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I.  ADMINISTRATIVE  AND  OTHER 
PROCEDURES 

1.  Assist  the  team  chief  in  organizing 
the  workload  for  the  105  facility 

2.  Perform  duties  of  team  chief  when 
the  team  chief  is  absent 

3.  Keep  up-to-date  log  on  the  410  and 
465  inn  times 

4.  Advise  team  chief  of  improperly 
handled  or  abused  items 

5.  Route  faulty  modules  through  the 
fault  isolation  and  repair  process 

6.  Prepare  and  maintain  the  required 
forms,  records  and  reports 

7.  Assist  maintenance  technicians  in 
in  preventative  maintenance  and 
and  repair  of  the  410 

8.  Use  the  technical  manuals  to 
operate  and  maintain  the  test 
equipment 

9.  Other  procedures  (specify) 
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6.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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J.  MAINTENANCE 
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1 .  Perform  preventative  maintenance 
checks  and  services  on  the  ETF 

2.  Repair  the  410  by  replacing 
assemblies  and  PCBs 

3.  Perform  preventative  maintenance 
checks  and  services  on  the 
oscilloscope 

4.  Assist  the  maintenance  technician 
in  performing  preventive 
maintenance  and  repairs  on  the 
410 

5.  Remove  and  replace  a  power 
supply 

6.  Replace  test  operators  panel 
lamp 

7.  Perform  maintenance  on  the 
Individual  stations 

a.  DC  Station 

b.  Control  Station 

c.  UUT  Station 

d.  PIU  Station 

e.  Tape  Station 

f.  465  DCT 

8.  Other  procedures  (specify) 
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Did  Not  Perform 


9.  Explanation  of  Borderline,  Difficult,  and  Very  Difficult 
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V.  COMPUTER  SOFTWARE 
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Using  the  scale  to  the  right, 
indicate  with  a  check  mark  (V) 
the  adequacy  of  the  computer 
software  in  each  of  the  following 
areas: 

1.  Type  of  user-software 
dialogue 

2.  Consistency  of  display 
formats 

3.  Clarity  of  prompts 

4.  Usefulness  of  prompts 

5.  Timeliness  of  prompts 

6.  Clarity  of  error 
messages 

7.  Usefulness  of  error 
messages 

8.  Timeliness  of  error 
messages 

9.  Clarity  of  test  procedures 
listed  in  the  software 

10.  Requirements  for  remembering 
related  information  while 
executing  a  program 


11.  Other  (specify) 
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Completely  Inadequate 


12.  Explanation  of  Borderline,  Mostly  Inadequate  and  Completely  Inadequate 
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VI.  SAFETY 


Indicate  with  a  check  mark  (a/)  whether  you  experienced  or  nearly 
experienced  any  of  the  following  safety  hazards  operating  the  ETF 


1.  Electrical  Shock 

2.  Burns 

3 .  Cut  or  Abrasions 

4.  Extreme  Brightness 

5.  Extreme  Loudness 

6.  Noxious  Fumes 


Nearly 

Neither 

Experienced  Nor 
Nearly 

Experienced  But 

Is  A 

Not 

A 

Experienced 

Experienced 

Hazard 

Hazard 

7.  Other  (specify)  _  _ 

8.  Explanation  of  the  above  items  which  received  checks  in  the 
“Experienced”,  “Nearly  Experienced”  and  “Is  a  Hazard”  categories. 


9.  Were  any  of  the  procedures  you  were  required  to  perform  during 
emplacement  or  displacement  of  the  system  unsafe  in  that  you  risked 
personal  harm  or  damage  to  the  facility  by  performing  them? 

(  )  Yes  (  )No 

Explanation  of  "Yes"  _ 
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Intentionally  left  blank. 
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Appendix  D.  Human  Factors  Engineering  Evaluation  of  Regency 

Net  Equipment 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Regency 

Human  Factors  Engineering  Evaluation  of  Regency  Net  Equipment 
Form  601-4-65 

Purpose:  The  items  on  this  form  ask  you  to  evaluate  various  human  factors 
aspects  of  the  equipment  you  use.  Given  this  and  other  information, 
corrective  action  can  be  taken  to  improve  equipment  design,  operator  training, 
task  requirements,  and  task  performance.  Only  Army  Research  Institute 
researchers  will  have  access  to  your  ratings.  Therefore,  please  be  honest  and 
objective  in  your  ratings. 

To  start  with:  Enter  you  PIN  number,  and  current  rank  in  the  spaces 
provided.  Circle  your  current  MOS.  Your  PIN  number  is  required  so  that  ARI 
research  personnel  may  contact  you  if  they  need  more  information  regarding 
your  responses. 

1.  PIN  No.  | _ | _ | _ |  2.  Rank:  | _ | _ | _ | 

3.  Circle  your  current  MOS:  3 1C  29S  52  29E  Other _ 

The  Regency  Net  system  is  composed  of  many  items  of  equipment,  such  as 
I/O  Units,  Vehicular  Adapters,  Receiver-Transmitters.  Operation  and 
maintenance  of  the  equipment  may  require  manual  or  electronic  adjustment  of 
different  parts  (e.g.,  display  screens,  touchpoints,  connectors,  cables, 
indicator  lights)  of  these  components.  On  each  of  the  following  pages  an 
equipment  component  will  be  identified,  and  several  human  factors  aspects  of 
the  component  will  be  noted.  Some  of  these  human  factors  aspects  may  be 
unsatisfactory,  needing  correction  or  modification;  some  may  be  satisfactory 
as  they  are.  Your  task  is  to  rate  how  satisfactory -unsatisfactory  each  Listed 
human  factor  is  for  the  component  named  on  each  page.  There  are  five  rating 
categories  available  for  your  use.  They  have  the  following  names  and 
abbreviations: 


RATING  CATEGORIES 

ABBREVIATION 

NUMERICAL 

VALUE 

Very  Unsatisfactory 

VU 

-2 

Unsatisfactory 

U 

-1 

Borderline 

B 

0 

Satisfactory 

S 

+1 

Very  Satisfactory 

VS 

+2 

The  rating  scale  below  contains  the  abbreviations  (to  save  space)  of  these 
five  categories.  The  purpose  of  the  numbers  is  to  show  that  the  ratings  change  from 
negative  to  positive,  with  a  0  under  B  (for  Borderline)  to  show  it  is  neither  negative  or 
positive.  The  underlines  give  you  a  place  on  which  to  mark  your  choice  of  the  five 
ratings. 


YU  U  B  S  VS 
-2  -1  0  +1  +2 


(Check  beneath  the  rating 
you  choose  to  give. 
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RATINGS  OF  DISPLAY  SCREEN  FACTORS 


Rating  Categories  VU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

4.  Brightness . 

5.  Glare . 


6.  Flicker. 


7.  Letter  clarity . 

8.  Viewing  distance.... 

9.  Angle  of  view . 

10.  Location  of  display 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  TOUCHPOINT  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

11.  Size . 


12.  Shape . 

13.  Spacing . 

14.  Label  understandability. 

15.  Label  relevance . 


16.  Label  location 


17.  Brightness 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  INDICATOR  LIGHT  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

19.  Brightness . 

20.  Glare . 


21.  Flicker 


22.  Viewing  distance . 

23.  Angle  of  view . 

24.  Label  understandability. 

25.  Label  correctness . 


26.  Label  visibility . 

27.  Location  of  indicators 


28.  Provides  needed  information 
in  a  timely  manner . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  CONNECTOR  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

29.  Brightness . 

30.  Glare . 


31.  Flicker 


32.  Letter  clarity . 

33.  Viewing  distance . 

34.  Angle  of  view . 

35.  Location  of  display . 

36.  Label  understandability. 

37.  Label  location . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  CABLE  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

38.  Coding  on  insulation 
(should  be  about  every 

12  inches)  .  _ 

39.  Routing  (for  easy  of 

inspection  &  maintenance) . 

40.  Positioned  to  avoid  damage 

by  tools,  hands,  feet .  .  .  .  .  . 

4 1 .  Safety  (protection  from 

sharp  edges) .  .  .  .  .  . 

42.  Mating  information  (labeled 
to  indicate  where  matings 

are  to  occur) .  .  .  .  .  . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  DISPLAY  SCREEN  FACTORS 


Rating  Categories  VU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

43.  Frequency  (too  high,  too  low) .  .  .  .  .  _ 

44.  Detectability . . .  . 

45.  Intensity .  .  .  .  .  . 

46.  Duration . 


47.  Ease  of  volume  adjustment . 

48.  Amount  of  volume 

adjustment  available . 

If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 


168 


RATINGS  OF  PUSHBUTTON  AND  SWITCH  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

49.  Shape  (concave  -  to  avoid 

finger  slipping) . 

50.  System  responsiveness 
(evidence  of  system 

response  when  pushed) . 

51.  Size . 


52.  Resistance  (too  hard, 

too  sensitive) . 

53.  Amount  of  displacement 

required . 

54.  Label 

understandability . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  TOGGLE  SWITCH  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

55.  Size . 


56.  Accidental  change  of 

switch  position? . 

57.  Distance  between 

on  and  off  positions . 

58.  System  responsiveness 
(evidence  of  system 
response  when  pushed) 

59.  Direction  of  movement 

(vertical  with  "down" 
to  turn  off) . 

60.  Labeling  (for  each 

position) . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  FUSE  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

61.  Positive  indication  of 

open  circuit . 

62.  Availability  of  special 
tools  to  remove  other 

components . 

63.  Identification  label  is 

on  equipment .  .  .  .  .  . 

64.  Rating  is  marked  on  fuse 

holder . 


65.  Label  legibility . 

66.  Ground  connection 


67.  Fan  operation 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  KEYPAD  FACTORS 


Rating  Categories  VU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

68.  Size . 


69.  Shape . 

70.  Spacing  of  keys . 

7 1 .  Resistance  (too  easy  or 

too  hard  to  push) . 

72.  Label  correctness . 


73.  Label  understandability 

74.  Label  location . 


75.  Brightness .  .  .  .  _ 

76.  System  responsiveness 

to  key  touching .  .  .  .  _  _ 

77.  Acceptability  of  keyboard . 

If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  HANDSET  AUDIO  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

78.  Ease  of  volume  adjustment . 

79.  Amount  of  volume 

adjustment . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  ROTARY  SWITCH  FACTORS 


Rating  Categories  VU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

80.  Shapes .  .  .  .  .  . 

81.  Position  (not  opposite 

each  other) . 

82.  Resistance  to  turning .  .  .  .  .  . 

83.  Pointer  visibility .  .  .  .  .  . 

84.  Parallax  (pointer  too  far 

from  scale?) .  .  .  .  .  . 

85.  Distance  between  index 

marks  on  scale . 


86.  System  responsiveness 
(to  movement  of  pointer) 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  VOLTAGE  METER  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

87.  Readable  (in  all  ambient 

lighting  conditions) . 

88.  Glare . 


89.  Legibility  of  numbers 

90.  Viewing  distance . 

91.  Label  location . 


92.  Location  of  meter. 


93.  Adjustability  (for  visual 
access) . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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RATINGS  OF  SLIDE  SWITCH  FACTORS 


Rating  Categories  YU  U  B  S  VS 

-2  -1  0  +1  +2 

VU  =  Very  Unsatisfactory 
U  =  Unsatisfactory 
B  =  Borderline 
S  =  Satisfactory 
VS  =  Very  Satisfactory 


VU  U  B  S  VS 
-2  -1  0  +1  +2 

94.  Protection  against 

accidental  activation . 


95.  Size 


96.  Resistance  pattern  when 

being  turned . 

97.  Distance  between  positions 

(RESET/OFF) . 

98.  System  responsiveness 

(to  movement  of  slide 
switch) . 

99.  Orientation  (vertical 
motion  downward  to 

turn  off) . 


If  any  of  the  above  human  factors  aspects  reduced  your  task  performance 
effectiveness,  a  brief  comment  about  it  and  its  effect  would  be  appreciated. 

Please  enter  the  item  number  in  front  of  your  comment. 
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Appendix  E.  Resource  Material  for  Health  Hazards,  System  Safety,  and 

Risk  Assessment  Matrix 


RESOURCE  MATERIAL  FOR  HEALTH  HAZARDS,  SYSTEM  SAFETY  AND  RISK 

ASSESSMENT  MATRIX 


Much  of  the  material  contained  in  this  section  has  been  abstracted  from  a  document 
entitled:  CLASSIFICATION  OF  DEFICIENCIES  AND  SHORTCOMINGS.  This  material 
represents  an  abstract  of  selected  sections  that  have  been  found  directly  relevant  to  implementing 
MANPRINT  assessment  in  these  domains.  In  this  abstraction  process,  material  is  abstracted 
from  the  whole  text  and  is  sometimes  quoted  in  total;  other  times  it  is  paraphrased.4 

1 .  Purpose  and  Scope.  This  material  describes  criteria  intended  to  provide  consistent  guidance 
for  classification  of  appropriate  material  test  incidents  and  test  results  as  deficiencies  and 
shortcomings  and  is  used  to  address  problems  identified  by  the  MANPRINT  Analysts  that  are 
generally  categorized  as  falling  within  the  Health  Hazards  and  System  Safety  domains.  While 
other  organizational  entities  have  primary  responsibility  for  certification  within  these  domains, 
the  MANPRINT  Analyst  reports  problems  falling  within  these  domains  that  he  observes  and/or 
have  been  reported  to  him  by  test  participants.  As  such,  the  basic  data  analyzed  by  the 
MANPRINT  Analyst  are  Test  Incident  Reports  (TIRs).  Occasionally  those  participants  are 
Subject  Matter  Experts  (SME);  most  often  they  are  soldiers  who  are  expected  to  operate  and 
maintain  the  system  under  test  once  it  is  fielded.  All  guidance  concerning  classification  of  test 
incidents  is  intended  for  use  in  classifying  deficiencies,  shortcomings,  suggested  improvements, 
and  other  incidents.  When  SMEs  are  used  this  intent  is  more  nearly  met  than  when 
representative  test  participants  are  providing  their  opinions.  As  such  opinions  and  judgments  are 
combined  to  provide  frequency  of  the  reported  problems.  Categorization  of  those  problems  is 
used  as  an  indicant  of  a  problem  that  may  require  further  study  by  the  Training  System  Manager 
prior  to  recommending  changes  in  the  material  system  or  operating  procedures. 

2.  TECHNICAL  PRESENTATION 

a.  Deficiency. 

1)  Definition:  A  deficiency  has  been  defined  as  a  defect  or  malfunction 
discovered  during  the  life  cycle  of  an  equipment  that  constitutes  a  safety 
hazard  to  personnel  that  will  result  in  serious  damage  to  the  equipment  if 
operation  is  continued  or  indicates  improper  design  or  other  cause  of  failure  of 
an  item  or  part  which  seriously  impairs  the  equipment’s  operational  capability. 
A  deficiency  normally  disables  or  immobilizes  the  equipment;  and  if 
occurring  during  test  phases,  will  serve  as  a  bar  to  type  classification  action. 


4 

Effort  has  been  to  focus  on  content  and  usability  of  the  material,  but  generally  there  has  been  no  conscious  effort 
made  to  attribute  specific  authorship  when  the  material  was  quoted  verbatim.  There  is  no  claim  or  representation  to 
indicate  that  the  author  of  this  Tutorial  document  claims  credit  for  the  specific  wording  when  quotes  are  used,  but 
not  credited.  This  liberty  has  been  taken  as  an  expedient  to  pull  together  a  sizeable  body  of  knowledge  in  a  short 
time.  In  those  cases,  the  author  of  this  document  requests  forgiveness  by  the  true  authors.  Material  contained  herein 
has  been  extracted  from  a  now  obsolete  document:  Test  Operations  Procedures  1-1-012,  CLASSIFICATION  OF 
DEFICIENCIES  AND  SHORTCOMINGS,  1  April  1979. 
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2)  Classification:  In  analysis  of  test  results  great  care  must  be  taken  to  insure 
proper  classification  of  a  test  incident  as  a  deficiency  or  a  shortcoming.  The  use  of  judgment, 
both  technical  and  military,  is  necessary  together  with  the  use  of  regulating  criteria  in  the 
analysis  of  test  incidents  before  classifying  them.  To  adequately  understand  the  fine  difference 
dividing  a  deficiency  from  a  shortcoming  it  is  necessary  to  examine  the  definitions  in  great 
detail.  In  order  for  a  test  incident  to  be  considered  a  deficiency  it  must  (1)  be  a  characteristic 
which  causes  the  failure,  not  the  failure  itself,  (2)  materially  and  seriously  degrade  the 
operational  capability  of  the  item,  (3)  not  be  the  result  of  an  isolate  failure,  (4)  apply  to  the 
population  from  which  the  sample  was  drawn,  and  (5)  be  stated  with  reasonable  certainty  that  it 
is  characteristic  of  the  items  under  test. 

b.  Shortcoming- 

Definition:  A  shortcoming  is  an  imperfection  or  malfunction  occurring  during  the 
life  cycle  of  equipment,  which  should  be  reported  and  which  must  be  corrected  to  increase 
efficiency  and  to  render  the  equipment  completely  serviceable.  I  will  not  cause  an  immediate 
breakdown,  jeopardize  safe  operation,  or  materially  reduce  the  usability  of  the  material  or  end 
product.  If  occurring  during  test  phases  the  shortcoming  should  be  corrected  if  it  can  be  done 
without  unduly  complicating  an  item  or  inducing  another  undesirable  characteristic  such  as 
increased  cost,  weight,  etc. 

c.  Suggested  Improvements. 

Definition:  A  suggested  improvement  is  an  increase  in  quality  or  performance 
which  is  desirable  but  not  imperative. 

d.  Hazard  Level  Categories.  MIL-STD-882  requires  that  safety  hazards  be  categorized  by 
both  severity  and  probability.  When  these  ratings  are  obtained  they  are  generally  asked  for 
through  separate  instruments  to  increase  the  likelihood  that  the  judgments  are  independent.  It  is 
not  appropriate  to  classify  all  catastrophic  and  critical  hazards  as  deficiencies  and  all  marginal 
hazards  as  shortcomings.  The  proper  classification  procedures  are  shown  in  Figure  2.  The 
following  definitions  apply  to  the  terms  used  in  that  figure. 

1)  Hazard  Severity.  A  qualitative  assessment  of  the  worst  potential 
consequences,  defined  by  the  degree  of  injury,  occupational  illness,  property  damage,  or 
equipment  damage  that  could  ultimately  occur. 

2)  Hazard  Probability.  The  likelihood,  expressed  in  quantitative  or  qualitative 
tenns,  that  a  hazard  will  occur. 
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Likely  to  occur 

Will  occur  several 

Likely  to  occur 
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Figure  1  Hazard  Probability  vs.  Hazard  Severity:  Risk  Assessment  Matrix 


Appendix  F.  Maneuver  Control  System  (MCS)  Training  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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MCS/P  IOT&E 


TRAINING  QUESTIONNAIRE 

June  24  1996 
Privacy  Act  Statement 

Authority;  5  USC  §  301,  Authority  for  the  Secretary  of  the  Army  to  Issue  Army  Regulations,  AR  73-1,  Test  and  Evaluation  Policy. 
Principal  Purpose:  The  data  to  be  collected  with  this  form  are  to  be  used  for  research  and  evaluation  purposes  only. 

Routine  Uses:  This  is  an  experimental  data  collection  questionnaire  developed  by  the  Test  and  Experimentation  Command  pursuant  to  its 
research  and  testing  mission  as  prescribed  in  AR  73-1 .  When  identifier  (name  and  social  security  number)  are  requested  they  are  to  be  used  for 
administrative  and  statistical  control  purposes  only.  Full  confidentiality  of  the  responses  will  be  maintained  in  the  processing  of  these  data. 

Disclosure:  Completion  of  this  questionnaire  is  voluntary.  You  are  encouraged  to  provide  complete  and  accurate  information  in  the  interests  of 
research  and  testing,  but  there  will  be  no  effect  on  individuals  for  not  providing  any  part  of  the  information. 

1. )  Date: _ 

2. )  Name: _ | _ 

(last  name,  first  name) 

3. )  Questionnaire  Serial  #:  TR _ 

4. )  Last  four  digits  of  your  social  security  number:  _ 

5. )  Duty  position  (mark  one  □):  □  operator  □  supervisor  □  staff  □  commander  □  other 

6. )  Training  Dates:  Start  (YYMMDD) _ End  (YYMMDD) _ 


Instructions 

The  purpose  of  this  questionnaire  is  to  get  your  opinions  about  MCS/P.  The  questionnaire  is  important  because  your 
answers  will  help  us  judge  how  well  MCS/P  has  been  built  for  soldiers.  Your  answers  will  not  be  given  to  or  shown 
to  anyone  except  those  who  are  evaluating  MCS/P  for  the  Army.  (For  example,  none  of  your  information  will  be 
given  to  your  chain  of  command  or  put  in  your  personnel  file)  Your  answers  will  be  treated  confidentially.  Please 
fill  out  the  questionnaire  carefully.  If  you  need  additional  space  to  answer  a  question,  indicate  by  an  arrow  (— >)  and 
continue  on  the  back  of  the  page.  Be  sure  to  number  the  item  on  the  back  of  the  page.  If  you  have  any  questions 
concerning  this  questionnaire,  please  contact  a  TEXCOM  representative  for  help.  Thank  you  for  your  help. 
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PART  1:  Training  Evaluation 

7. )  Which  type  of  training  are  you  evaluating?  (mark  one  □  ) 

□  Staff  User  (SU)  □  System  Manager  (SM)  □  System  Maintainer  □  Senior  Staff  (SS)  □  Executive  (EX) 

8. )  The  amount  of  time  spent  on  hands-on  training  was  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

9.  The  amount  of  time  spent  you  had  to  ask  questions  was  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

10. )  Did  this  training  train  you  to  a  level  that  allowed  you  to  operate  the  equipment  without  help  from  someone 
else?  (mark  one  □  ): 

□  yes  □  no  — >  please  explain: 


11. )  The  use  of  training  support  materials  (TMs,  handouts,  videos,  Vu-graph  slides)  was  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

12. )  The  opportunities  to  practice  skills  with  other  operators  was  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

13. )  The  opportunities  to  get  help  from  instructors  was  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

14. )  The  content  of  instructor  handouts  was  (mark  one  □): 

□  very  appropriate  □  appropriate  □  borderline  □  inappropriate  □  very  inappropriate 
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15.  The  format  of  instructor  handouts  was  (mark  one  □): 


□  very  appropriate  □  appropriate  □  borderline  □  inappropriate  □  very  inappropriate 

16.  The  content  of  Vu-graph  slides  (overhead  slides)  was  (mark  one  □): 

□  very  appropriate  □  appropriate  □  borderline  □  inappropriate  □  very  inappropriate 

17. )  The  format  of  Vu-graph  slides  (overhead  slides)  was  (mark  one  □): 

□  very  appropriate  □  appropriate  □  borderline  □  inappropriate  □  very  inappropriate 

18. )  Do  the  guide  sheets  for  practical  exercises  make  sense?  (mark  one  □): 

□  yes  □  no  — »  please  explain: _ 


19.)  Are  critical  (primary)  tasks  clearly  identified  (mark  one  □): 
□  yes  □  no  — >  please  explain: _ 


20.)  Are  the  training  goals  of  this  training  clearly  stated?  (mark  one  □): 

□  yes  □  no  — >  please  explain: _ 


21.)  The  standards  used  to  evaluate  your  progress  were  (mark  one  □): 

□  very  appropriate  □  appropriate  □  borderline  □  inappropriate  □  very  inappropriate 
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22.)  Did  you  have  a  chance  to  practice  new  skills  after  they  were  taught  (practice  did  not  have  to  wait  until  many 
new  skills  were  presented?  (mark  one  □): 

□  yes  □  no  — »  please  explain: _ 


23. )  Training  time  was  used  (mark  one  □): 

□  very  effectively^  effectively  □  borderline  □  ineffectively  □  very  ineffectively 

24. )  The  instruction  on  the  use  of  equipment  TMs  was  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

25. )  System  safety  issues  addressed  in  class  were  (mark  one  □): 

□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

26. )  Is  the  software  version  trained  the  same  version  installed  on  the  equipment  used  in  class? 
(mark  one  □): 

□  yes  □  no  — »  please  explain: _ 


27. )  The  pace  at  which  new  material  is  presented  in  class  is  (mark  one  □): 

□  very  appropriate  □  appropriate  □  borderline  □  inappropriate  □  very  inappropriate 

28. )  Were  you  allowed  to  practice  all  new  or  updated  tasks  in  class?  (mark  one  □): 

□  yes  □  no  — >  please  explain: _ 
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29.)  When  compared  to  the  length  of  class  time,  the  number  of  new  tasks/procedures  was  (mark  one  □): 


□  very  adequate  □  adequate  □  borderline  □  inadequate  □  very  inadequate 

30.)  Was  refresher  training  on  old  tasks  and  procedures  provided?  (mark  one  □): 

□  yes  □  no  — »  please  explain: _ 


31.)  After  you  completed  the  training,  do  you  have  the  ability  to  perform  all  critical  tasks  and  procedures? 
(mark  one  □): 

□  yes  □  no  — >  please  explain: _ 


PART  II:  Training  Devices 

Used  During  Training?  Time  Spent 


Training  Device 
Technique  or  Topic 

NO 

YES 

Too 

Much 

About 

Right 

Too 

Little 

32.)  Lecture  and 
Discussion 

□ 

□  —>—>->• 

□ 

□ 

□ 

33.)  Hands-on 
Training 

□ 

□ 

□ 

□ 

34.)  Refresher 
Training  on 

Old  Tasks 

□ 

” 

□ 

□ 

□ 

35.)  Practical 
Exercises 

□ 

□ 

□ 

□ 

36.)  Instruction  of 
Critical  Tasks 

□ 

□ 

□ 

□ 

37. )  Discussion  of 

Safety  Issues 

38. )  Actual  Equip¬ 

□ 

□  — >— »  — > 

□ 

□ 

□ 

ment  to  be 
Operated  in 

Unit 

□ 

□ 

□ 

□ 
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PART  II:  Training  Devices  (continued) 


Used  During  Training?  Time  Spent 


Training  Device 
Technique  or  Topic 

NO 

YES 

Too 

Much 

About 

Right 

Too 

Little 

39.)  Video  Tapes 

□ 

□ 

□ 

□ 

40.)  Technical 

□ 

□ 

□ 

□ 

Manuals 

□ 

□ 

□ 

□ 

42.)  Vu-graphs 

□ 

□ 

□ 

□ 

(overhead  slides) 
43.)  Computer  Screen 

□ 

□  —>—>—>■ 

□ 

□ 

□ 

Projector 

44.)  Actual  Commun¬ 

□ 

□ 

□ 

□ 

ications  Network 
45.)  Total  Amount  of 
Training  Time 

□ 

□ 

□ 

PART  III:  MCS/P  Tasks 


KEY :  M=Need  Much  More,  SM=Need  Some  More,  R=About  Right,  SL=Need  Some  Less,  L=Need  Much  Less 

Task  Training  Time/ 

Trained  Task  Coverage 


t  asK  or  t operation _ 

46.)  Perform  MCS/P 
and  SICPS 
Maintenance 

_ INO 

□ 

Yes 

□  — > 

IVI 

□ 

MVI 

□ 

_ Li _ 

□ 

M- 

□ 

_ L _ 

□ 

47)  Prepare  MCS/P 
for  Operations 

48.)  Perform  PMCS 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

on  the  MCS/P 
Equipment 

49.)  Prepare  MCS/P 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

for  Movement 

50.)  Troubleshoot/ 

Fault  Isolate  the 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

MCS/P 

Equipment 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

51.)  Employ  a  SICPS 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 
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PART  III:  MCS/P  Tasks  (continued) 


KEY :  M=Need  Much  More,  SM=Need  Some  More,  R=About  Right,  SL=Need  Some  Less,  L=Need  Much  Less 


Task  Training  Time/ 

Trained  Task  Coverage 


Task  or  Operation 

No 

Yes 

M 

SM 

R 

SL 

L 

52. )  Perform  PMCS  on  the 

SICPS  Equipment 

53. )  Troubleshoot/ Fault 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

Isolate  the  SICPS 
Equipment 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

54.)  Prepare  the  SICPS  for 

Movement 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

55.)  Troubleshoot/Fault 

Isolate  and  Restore 
the  LAN 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

56. )  Perform  Information 

Security  Functions 

57. )  Perform  System 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

Administration  Functions 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

58.)  Operate  UTO  Software 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

59.)  Operate  Reports 

Software 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

60.)  Operate  Message 

Software 

□  — > 

□ 

□ 

□ 

□ 

□ 

61.)  Perform  System  Manager 

□ 

□ 

□ 

□ 

□ 

Functions 

□  — > 

62.)  Operate  OPORD 

Software 

□ 

□  — ► 

□ 

□ 

□ 

□ 

□ 

63. )  Operate  Maps  and 

Overlays  Software 

64. )  Perform  Briefing  System 

□ 

□  — ► 

□ 

□ 

□ 

□ 

□ 

Functions 

□ 

□  — ► 

□ 

□ 

□ 

□ 

□ 

65.)  Perform  Telestrator 

Functions 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

66. )  Plan  for  CONOPS 

67. )  Plan  the  MCS/P  Network 

□ 

□  — » 

□ 

□ 

□ 

□ 

□ 

Architecture 

□ 

□  — > 

□ 

□ 

□ 

□ 

□ 

68.)  TEST  SCORES  (to  be  provided  by  TEXCOM): 

SM  PT  #1: _  SM  PT  #2: _ 

SS  WE:  SS  PE: 
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Appendix  G.  Maneuver  Control  System  Workload  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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MCS/P  IOT&E 


WORKLOAD  QUESTIONNAIRE 

June  24  1996 
Privacy  Act  Statement 

Authority;  5  USC  §  301,  Authority  for  the  Secretary  of  the  Army  to  Issue  Army  Regulations,  AR  73-1,  Test  and  Evaluation  Policy. 
Principal  Purpose:  The  data  to  be  collected  with  this  form  are  to  be  used  for  research  and  evaluation  purposes  only. 

Routine  Uses:  This  is  an  experimental  data  collection  questionnaire  developed  by  the  Test  and  Experimentation  Command  pursuant  to  its 
research  and  testing  mission  as  prescribed  in  AR  73-1 .  When  identifier  (name  and  social  security  number)  are  requested  they  are  to  be  used  for 
administrative  and  statistical  control  purposes  only.  Full  confidentiality  of  the  responses  will  be  maintained  in  the  processing  of  these  data. 

Disclosure:  Completion  of  this  questionnaire  is  voluntary.  You  are  encouraged  to  provide  complete  and  accurate  information  in  the  interests  of 
research  and  testing,  but  there  will  be  no  effect  on  individuals  for  not  providing  any  part  of  the  information. 

1. )  Date: _ 

2. )  Rank: _ 

3. )  Name: _ | _ 

(last  name,  first  name) 

4. )  Questionnaire  Serial  #:  WL  _ 

5. )  Last  four  digits  of  your  social  security  number:  _ 

6. )  Duty  position  (mark  one  □):  □  staff  user  □  system  manager  □  staff  □  commander 


Instructions:  Using  the  definitions  provided  below,  darken  the  appropriate  circle  on  the  scale 
For  each  aspect  for  each  task  listed.  Please  darken  only  in  one  of  the  circles  provided.  The 
Results  of  this  questionnaire  will  be  used  to  help  the  U.S.  Army  improve  MCS/P  for  your  use. 

Thanks  for  the  help. 

Definitions  of  the  six  aspects  you  will  be  using  with  rating  scales: 


Aspect  Description 

Mental  Demand .  How  mentally  demanding  was  the  task? 

Physical  Demand .  How  physically  demanding  was  the  task? 

Temporal  Demand .  How  hurried  or  rushed  was  the  pace  of  the  task? 

Performance .  How  successful  were  you  in  accomplishing  what 

you  were  asked  to  do? 

Effort . How  hard  did  you  have  to  work  to  accomplish 

your  level  of  performance? 

Frustration .  How  insecure,  discouraged,  irritated,  and  annoyed 

were  you? 
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1.  Perform  MCS/P  and  SICPS  Maintenance. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


2.  Prepare  MCS/P  for  Operations. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ©- 


-© - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand 


® - O - © 


Very 

Low 


-© — o — ©- 


-© — o — ®- 


-@ — o — ® 


Very 

High 


Performance 


® - O — ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 
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3.  Perform  PMCS  on  the  MCS/P  Equipment. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ®- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


4.  Prepare  MCS/P  for  Movement. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ©- 


-®- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ©- 


-® - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-© — o — ®- 


— o — ® 

Very 

High 


Performance 


® - O — ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 
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5.  Troubleshoot/Fault  Isolate  the  MCS/P  Equipment. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


6.  Employ  a  SICPS 


Mental  Demand 


-© - O - ®- 


-© - O - ©- 


-© — o — ®- 


Low 


Very 


Physical  Demand 


-© - O - ©- 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Temporal  Demand  ®- 
Very 
Low 


-© — o — ©- 


-© — o — ®- 


® — o — ® 


Very 

High 


Performance 


-© - O - ®- 


-© - O - ©- 


Very 

Low 


-® — o — ®- 


-® — o — ® 


Very 

High 


Effort 


-© - O - ©- 


Very 

Low 


-® — o — ®- 


-® — o — ® 


Very 

High 


Fmstration 


-© - O - ©- 


Very 

Low 


-® — o — ®- 


-® — o — ® 


Very 

High 
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7.  Perform  PMCS  on  the  SICPS  Equipment. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ®- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


8.  Troubleshoot/Fault  Isolate  SICPS  Equipment- 


Mental  Demand  ®- 

Very 

Low 


-®- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ©- 


-® - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-© — o — ®- 


— o — ® 

Very 

High 


Performance 


® - O — ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 
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9.  Prepare  the  SICPS  for  Movement. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


10.  Troubleshoot/Fault  Isolate  and  Restore  the  LAN. 

Mental  Demand  ® — O — © — O — © — O — © — O — ©- 

Very 
Low 


-© — o — ®- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ©- 


-© - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-© — o — ©- 


-© — o — ®- 


-@ — o — ® 

Very 

High 


Performance 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Page  6  of  12 


195 


11.  Perform  Information  Security  Functions. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ©- 


Very 

High 


Temporal  Demand 


® - O - © 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ® 


Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


12.  Perform  System  Administration  Functions. 


Mental  Demand  ®- 

Very 

Low 


D — o — ® — o — © — o — ®- 


High 


Very 


Physical  Demand  ® — 

Very 

Low 


-® - O - ®- 


-© - O - ©- 


-®- 


Very 

High 


Temporal  Demand 


® - O - © 


Very 

Low 


-© — o — ©- 


-© — o — ®- 


-@ — o — ® 


Very 

High 


Performance 


® - O — ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 
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13.  Operate  UTO  Software. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-© — o — ®- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ®- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ®- 


Very 

High 


Frastration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


14.  Operate  Reports  Software. 


Mental  Demand  ®- 

Very 

Low 


-© — o — © — o — © — o — © — o — ® — o — ®- 


High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ® - O - © - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-© — o — ©- 


-© — o — ®- 


-© — o — ® 

Very 

High 


Performance 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Page  8  of  12 


197 


15.  Operate  Message  Software. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-© — o — ®- 


Very 

High 


Temporal  Demand 


® - O - © 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ® 


Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


16.  Perform  System  Manager  Functions. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ©- 


-©- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-® - O - ©- 


-® - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-© — o — ©- 


-® — o — ®- 


-© — o — ® 

Very 

High 


Performance 


® - O — ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 
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17.  Operate  OPLAN/OPORD  Software. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-© — o — ®- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ®- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ®- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Frastration 


® - O - ®- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


18.  Plan  the  MCS/P  Network  Architecture. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ®- 


High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ©- 


-© - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-© — o — ©- 


-© — o — ®- 


-© — o — ® 

Very 

High 


Performance 


® - O — ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 
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19.  Plan  for  CONOPS. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ®- 


D — o — ®- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 
Very 
Low 


-© — o — ©- 


-® — o — ©- 


-®- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-© — o — ®- 


-© — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ®- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ®- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


20.  Execute  CONOPS. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ® — o — @ — o — © — o — © — o — © — o — © — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-© - O - ® - O - © - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-© — o — ®- 


-© — o — ©- 


-© — o — ®- 


-© — o — ® 

Very 

High 


Performance 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-® — o — ®- 


-® — o — ®- 


Very 

High 


Frustration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 
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21.  Perform  Briefing  System  Functions. 


Mental  Demand  ®- 

Very 

Low 


Very 


Physical  Demand  ®- 

Very 

Low 


-® — o — ©- 


-® — o — ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-® — o — ©- 


-® — o — ®- 


-® — o — ® 

Very 

High 


Performance 


® - O - ©- 


Very 

Low 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 


Very 

Low 


-© — o — ©- 


-® — o — ®- 


Very 

High 


Fmstration 


® - O - ®- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 


22.  Perform  Telestrator  Functions. 


Mental  Demand  ®- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ® — 

High 


Very 


Physical  Demand  ® — 

Very 

Low 


-® - O - ©- 


-© - O - ©- 


-®- 


-® - O - ©- 


Very 

High 


Temporal  Demand  ® - O — ©- 

Very 

Low 


-® — o — ®- 


-© — o — ©- 


-© — o — ®- 


-@ — o — ® 

Very 

High 


Performance 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Effort 


® - O - ©- 

Very 

Low 


-© — o — ©- 


-© — o — ®- 


-® — o — ®- 


Very 

High 


Fmstration 


® - O - ©- 


Very 

Low 


-© — o — ®- 


-® — o — ®- 


Very 

High 
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Intentionally  left  blank. 
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Appendix  H.  MANPRINT  Survey/Interview  Questions  for  All-Source 
Analysis  System  (ASAS)  Block  II  Initial  Operational  Test 
and  Evaluation  (IOTE) 


This  appendix  appears  in  its  original  form,  without  editorial  change. 


Analysis  and  Control  Element  (ACE)  IOTE 
MANPRINT  SURVEY/INTERVIEW  QUESTIONS 


Instructions: 


1 .  Please  check  the  Functional  Identities  on  which  you  have  been  serving  as  an  Operator  and  indicate  your  Echelon 
(e.g.,  Btn,  Bde,  Div)  of  operation.  It  is  understood  that  some  soldiers  will  be  operating  in  more  than  one 
functionality. 


BIO/BACKGROUND/POSITION 


Name 

PIN# 

Date 

(Please  Print) 

(Last  4  SSN) 

MM/DD/YY 

Functional  Identities  (ACE  Components)  on  which  you  will  be  serving  as  a  Test  Participant  Operator  (please  check): 

_  ISS 

_ccs 

_  Shared  SS 
_  COMINT 
_  IMINT 
_  RIS/ELINT 
_  SIGINT/MASINT 
_  CI/HUMINT 

_ OSINT 

_AS 

_SIT/IPB/DB 
_  TGTDEV 

_ CM/ISR 

_ Asset  Mgmnt 

_  Trusted  Suite  (TWS) 

Echelon  (Btn,  Bde,  Div)  _ 


2.  This  instrument  contains  a  series  of  Statements.  Statements  1-9  focus  on  Equipment  ( items  a-n)  within  the  ACE. 
Statements  10-18  focus  on  Functions  ( items  al-tl)  of  the  ACE. 

3.  Statements  differ  only  in  the  MANPRINT  domain  to  which  they  refer.  The  domain  is  underlined  in  the  stem  of 
each  statement. 

4.  As  you  insert  each  item  within  each  stem  to  form  a  statement,  ask  yourself  whether  this  is  true  (Y)  or  false  (N) 
and  record  the  answer  in  the  column  to  the  left  of  the  item.  Put  Not  Applicable  (N/A)  if  you  did  not  operate  this 
piece  of  Equipment  (or  perform  this  Function)  in  the  ACE. 

5.  For  those  cases  where  you  recorded:  true  (Y),  please  make  a  Rating  using  the  scale  shown  at  the  top  of  each  page 
to  record  your  amount  of  agreement  with  the  Statement.  Ratings  should  be  recorded  in  the  right  hand  column  after 
each  item. 

6.  IN  CASES  WHERE  YOU  INDICATE  TRUE  (Y),  AN  EXPLANATION  "WHY"  IS  NEEDED.  PLEASE 
WRITE  YOUR  RESPONSE  ON  THE  BACK  OF  THIS  FORM  AND  INDICATE  FOR  WHICH  QUESTION  YOU 
ARE  MAKING  COMMENT. 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Equipment  Focus:  Health  Hazards 

Y/N 

1 .  Performing  tasks  or  operations  with _ jeopardized  my 

safety  or  that  of  another  Team  member. 

_  a.  TROJAN  special  purpose  integrated  remote  intelligence 

tenninal  (SPIRIT) 

_  b.  Tactical  communications  support  processor  (TCSP) 

_  c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 

(TPN) 

_  d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

_  e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

_  f.  Secure  messaging  and  routing  terminal  (SMART) 

_  g.  Multi  function  work  stations 

_  h.  Remote  intelligence  server 

_  i.  Trusted  workstation 

_  j.  Web  guards 

_  k.  Web  shields 

_  1.  Trusted  Guard  System 

_  m.  Tactical  image  processing  subsystem  (TIPS) 

_  n.  Moving  target  indicator  module 


Rating 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Y/N 


Equipment  Focus:  Health  Hazard 

2.  Have  experienced  or  suspected  one  or  more  conditions 

associated  with  operating  the _ which  I  considered 

unhealthy. 

a.  TROJAN  special  purpose  integrated  remote  intelligence 
terminal  (SPIRIT) 

b.  Tactical  communications  support  processor  (TCSP) 

c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 
(TPN) 

d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

f.  Secure  messaging  and  routing  terminal  (SMART) 

g.  Multi  function  work  stations 

h.  Remote  intelligence  server 

i.  Trusted  workstation 

j.  Web  guards 

k.  Web  shields 

l.  Trusted  Guard  System 

m.  Tactical  image  processing  subsystem  (TIPS) 

n.  Moving  target  indicator  module 


Rating 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 

3  =  Generally  Disagree  5  =  Strongly  Agree 

4  =  Generally  Agree  6  =  Completely  Agree 

9  =  Unknown 

Y/N 

Equipment  Focus:  System  Safety 

Rating 

3 .  Have  noted  safety  problems  (actual  or  potential)  related  to  operation  of 


a.  TROJAN  special  purpose  integrated  remote  intelligence  tenninal 
(SPIRIT) 

b.  Tactical  communications  support  processor  (TCSP) 

c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 
(TPN) 

d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

f.  Secure  messaging  and  routing  terminal  (SMART) 

g.  Multi  function  work  stations 

h.  Remote  intelligence  server 

i.  Trusted  workstation 

j.  Web  guards 

k.  Web  shields 

l.  Trusted  Guard  System 

m.  Tactical  image  processing  subsystem  (TIPS) 

n.  Moving  target  indicator  module 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Equipment  Focus:  Training 

Y/N 

4.  Have  been  required  to  perform  one  or  more  tasks  or  operations 
with _ for  which  your  training  was  poor  or  inadequate. 

a.  TROJAN  special  purpose  integrated  remote  intelligence 
_  tenninal  (SPIRIT) 

_  b.  Tactical  communications  support  processor  (TCSP) 

_  c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 

(TPN) 

_  d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

_  e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

_  f.  Secure  messaging  and  routing  terminal  (SMART) 

_  g.  Multi  function  work  stations 

_  h.  Remote  intelligence  server 

_  i.  Trusted  workstation 

_  j.  Web  guards 

_  k.  Web  shields 

_  1.  Trusted  Guard  System 

_  m.  Tactical  image  processing  subsystem  (TIPS) 

_  n.  Moving  target  indicator  module 


Rating 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Y/N 


Equipment  Focus:  Manpower 

5.  Have  been  required  to  perforin  one  or  more  tasks  or  operations  with 
_ that  required  an  additional  Team  Member. 

a.  TROJAN  special  purpose  integrated  remote  intelligence 
tenninal  (SPIRIT) 

b.  Tactical  communications  support  processor  (TCSP) 

c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 
(TPN) 

d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

f.  Secure  messaging  and  routing  terminal  (SMART) 

g.  Multi  function  work  stations 

h.  Remote  intelligence  server 

i.  Trusted  workstation 

j.  Web  guards 

k.  Web  shields 

l.  Trusted  Guard  System 

m.  Tactical  image  processing  subsystem  (TIPS) 

n.  Moving  target  indicator  module 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Equipment  Focus:  Personnel 


Y/N 


Rating 


6.  Have  been  required  to  perforin  one  or  more  tasks  or  operations 

with _ that  was  outside  of  "normal"  duties  for  my  MOS  and 

skill  level. 

a.  TROJAN  special  purpose  integrated  remote  intelligence  tenninal 
(SPIRIT) 

b.  Tactical  communications  support  processor  (TCSP) 

c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 
(TPN) 

d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

f.  Secure  messaging  and  routing  terminal  (SMART) 

g.  Multi  function  work  stations 

h.  Remote  intelligence  server 

i.  Trusted  workstation 

j.  Web  guards 

k.  Web  shields 

l.  Trusted  Guard  System 

m.  Tactical  image  processing  subsystem  (TIPS) 

n.  Moving  target  indicator  module 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Equipment  Focus:  Human  Factors  Engineering— Procedures 

Y/N 

7.  Have  found  procedures  related  to  operation  of _ that 

were  unnecessarily  difficult,  complicated  or  "unfriendly. 

_  a.  TROJAN  special  purpose  integrated  remote  intelligence  tenninal 

(SPIRIT) 

_  b.  Tactical  communications  support  processor  (TCSP) 

_  c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 

(TPN) 

_  d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

_  e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

_  f.  Secure  messaging  and  routing  terminal  (SMART) 

_  g.  Multi  function  work  stations 

_  h.  Remote  intelligence  server 

_  i.  Trusted  workstation 

_  j.  Web  guards 

_  k.  Web  shields 

_  1.  Trusted  Guard  System 

_  m.  Tactical  image  processing  subsystem  (TIPS) 

_  n.  Moving  target  indicator  module 


=  Unknown 


Rating 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Equipment  Focus:  Human  Factors  Engineering— Software 

Y/N 

8.  Have  found  software  problems  associated  with  operation  of 

_  a.  TROJAN  special  purpose  integrated  remote  intelligence  tenninal 

(SPIRIT) 

_  b.  Tactical  communications  support  processor  (TCSP) 

_  c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 

(TPN) 

_  d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

_  e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

_  f.  Secure  messaging  and  routing  terminal  (SMART) 

_  g.  Multi  function  work  stations 

_  h.  Remote  intelligence  server 

_  i.  Trusted  workstation 

_  j.  Web  guards 

_  k.  Web  shields 

_  1.  Trusted  Guard  System 

_  m.  Tactical  image  processing  subsystem  (TIPS) 

_  n.  Moving  target  indicator  module 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Equipment  Focus:  Human  Factors  Engineering— Placement 

Y/N 

9.  Have  found  that  placement  of _ (or  their  switches  or 

indicators)  made  access  or  use  difficult. 

_  a.  TROJAN  special  purpose  integrated  remote  intelligence  terminal 

(SPIRIT) 

_  b.  Tactical  communications  support  processor  (TCSP) 

_  c.  Multiple  Subscriber  Equipment  (MSE)  tactical  packet  network 

(TPN) 

_  d.  Direct  wire  line  connection  to  AN/TYC-39  message  switch 

_  e.  Ultra-high  frequency  net  radio  protocol  (UHF-NRP) 

_  f.  Secure  messaging  and  routing  terminal  (SMART) 

_  g.  Multi  function  work  stations 

_  h.  Remote  intelligence  server 

_  i.  Trusted  workstation 

_  j.  Web  guards 

_  k.  Web  shields 

_  1.  Trusted  Guard  System 

_  m.  Tactical  image  processing  subsystem  (TIPS) 

_  n.  Moving  target  indicator  module 


=  Unknown 


Rating 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Function  Focus:  Health  Hazards 

Y/N 

10.  Perfonning _ jeopardized  my  safety  or  that  of  another 

Team  Member. 

_  al .  communications  front  end  with  the  CCS 

_  b  1 .  an  interface  between  Block  II  ACE  All  Source  and 

communication  systems  with  the  CCS 

_  c  1 .  an  interface  between  Block  II  ACE  Single  Source  and 

communication  systems  with  the  CCS 

_  d  1 .  an  interface  between  Block  II  ACE  All  Source  and  forward 

tactical  sensors  with  the  CCS 

_  el .  an  interface  between  Block  II  ACE  Single  Source  and  forward 

tactical  sensors  with  the  CCS 

_  fl .  receipt  and  relay  of  information  between  (vertical)  and  within 

(horizontal)  echelons  with  the  CCS 

_  gl.  multi-level  security  (MLS)— SCI  and  collateral  traffic  with  the 

CCS 

_  hi.  support  to  COMINT 

_  il .  support  to  ELINT 

_  j  1 .  support  to  CI/HUMINT 

_  kl.  support  to  Measurement  and  Signature  Intelligence  (MASINT) 

_  11 .  support  to  Open  Source  Intelligence  (OSINT) 

_  ml .  support  to  All  Source  Fusion 

_  nl .  support  to  Situation  Development 

_  o  1 .  support  to  IPB 

_  p  1 .  support  to  Target  Identification 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 


Function  Focus:  Health  Hazards 


Y/N 


Rating 


10  (cont.)  Perfonning _ jeopardized  my  safety  or  that  of 

another  Team  Member. 

ql.  support  to  Target  Refinement 

rl.  support  to  Target  Nomination 

si.  support  to  ISR  requirements 

tl.  support  to  Asset  Management  Responsibilities 


215 


1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Function  Focus:  Health  Hazards 


Y/N 


Rating 


1 1 .  Have  experienced  or  suspected  one  or  more  conditions 
associated  with _ which  I  considered  unhealthy 

al.  using  the  CCS  for  front  end  communications 

bl.  using  the  CCS  to  interface  between  Block  II  ACE  All  Source 
and  communication  systems  with  the  CCS 

cl.  using  the  CCS  to  interface  between  Block  II  ACE  Single  Source 
and  communication  systems 

dl.  using  the  CCS  to  interface  between  Block  II  ACE  All  Source 
and  forward  tactical  sensors 

el.  using  the  CCS  to  interface  between  Block  II  ACE  Single  Source 
and  forward  tactical  sensors 

fl .  using  the  CCS  to  receive/relay  of  information  between  (vertical) 
and  within  (horizontal)  echelons 

gl.  using  the  CCS  to  control  multi-level  security  (MLS)— SCI  and 
collateral  traffic 

hi.  using  the  ACE  to  support  COMINT 

i  1 .  using  the  ACE  to  support  ELINT 

j  1 .  using  the  ACE  to  support  CI/HUMINT 

kl.  using  the  ACE  to  support  Measurement  and  Signature 
Intelligence  (MASINT) 

11.  using  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml.  using  the  ACE  to  support  All  Source  Fusion 
nl.  using  the  ACE  to  support  Situation  Development 
o  1 .  using  the  ACE  to  support  IPB 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Y/N 


Function  Focus:  Health  Hazards 


1  l(cont)  Have  experienced  or  suspected  one  or  more  conditions 
associated  with _ which  I  considered  unhealthy 

pi.  using  the  ACE  to  support  Target  Identification 

ql.  using  the  ACE  to  support  Target  Refinement 

rl.  using  the  ACE  to  support  Target  Nomination 

si.  using  the  ACE  to  support  ISR  requirements 

tl.  using  the  ACE  to  support  Asset  Management  Responsibilities 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  System  Safety 


Y/N 


Rating 


12.  Have  noted  system  safety  problems  (actual  or  potential)  related  to 
operation  of _ . 

al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
forward  tactical  sensors 


fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 
traffic 

hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml .  the  ACE  to  support  All  Source  Fusion 
nl.  the  ACE  to  support  Situation  Development 
o  1 .  the  ACE  to  support  IPB 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  System  Safety 


Y/N  Rating 

12(cont)  Have  noted  system  safety  problems  (actual  or  potential)  related 
to  operation  of _ . 

_  pi.  the  ACE  to  support  Target  Identification  _ 

_  ql.  the  ACE  to  support  Target  Refinement  _ 

_  rl .  the  ACE  to  support  Target  Nomination  _ 

_  si .  the  ACE  to  support  ISR  requirements  _ 

_  tl .  the  ACE  to  support  Asset  Management  Responsibilities  _ 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Training 


Y/N 


Rating 


13.  Have  been  required  to  perfonn  one  or  more  tasks  or  operations 
with _ for  which  my  training  was  poor  or  inadequate. 

al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
forward  tactical  sensors 


fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 
traffic 

hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml .  the  ACE  to  support  All  Source  Fusion 
nl.  the  ACE  to  support  Situation  Development 
o  1 .  the  ACE  to  support  IPB 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Training 


Y/N 


Rating 


13(cont).  Have  been  required  to  perform  one  or  more  tasks  or 

operations  with _ for  which  my  training  was  poor 

or  inadequate 

pi.  the  ACE  to  support  Target  Identification 

ql.  the  ACE  to  support  Target  Refinement 

rl.  the  ACE  to  support  Target  Nomination 

si.  the  ACE  to  support  ISR  requirements 

tl .  the  ACE  to  support  Asset  Management  Responsibilities 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Manpower 


Y/N 


Rating 


14.  Have  been  required  to  perfonn  one  or  more  tasks  or  operations 
with _ that  required  an  additional  Team  Member. 

al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
forward  tactical  sensors 


fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 
traffic 

hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml .  the  ACE  to  support  All  Source  Fusion 
nl.  the  ACE  to  support  Situation  Development 
o  1 .  the  ACE  to  support  IPB 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Manpower 


Y/N 


Rating 


14  (cont).  Have  been  required  to  perform  one  or  more  tasks  or 

operations  with _ that  required  an  additional  Team 

Member. 

pi.  the  ACE  to  support  Target  Identification 

ql.  the  ACE  to  support  Target  Refinement 

rl.  the  ACE  to  support  Target  Nomination 

si.  the  ACE  to  support  ISR  requirements 

tl .  the  ACE  to  support  Asset  Management  Responsibilities 


223 


1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Function  Focus:  Personnel 


Y/N 


Rating 


15.  Required  to  perform  tasks  or  operations  with  _ 

that  was  outside  of  "normal"  duties  for  my  MOS  and  skill  level. 

al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source 
and  forward  tactical  sensors 

fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 

traffic 


hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml .  the  ACE  to  support  All  Source  Fusion 
nl.  the  ACE  to  support  Situation  Development 
o  1 .  the  ACE  to  support  IPB 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Personnel 


Y/N 


15  (cont)  Required  to  perform  tasks  or  operations  with _ 

that  was  outside  of  "normal"  duties  for  my  MOS  and  skill  level. 

pi.  the  ACE  to  support  Target  Identification 


ql.  the  ACE  to  support  Target  Refinement 

rl.  the  ACE  to  support  Target  Nomination 

si.  the  ACE  to  support  ISR  requirements 

tl .  the  ACE  to  support  Asset  Management  Responsibilities 


9  =  Unknown 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Human  Factors  Engineering— Procedures 


Y/N 


Rating 


16.  Have  found  procedures  related  to  operation  of _ 

that  were  unnecessarily  difficult,  complicated  or  "unfriendly". 

al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
forward  tactical  sensors 


fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 
traffic 

hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml .  the  ACE  to  support  All  Source  Fusion 
nl.  the  ACE  to  support  Situation  Development 
o  1 .  the  ACE  to  support  IPB 


226 


1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Human  Factors  Engineering— Procedures 


Y/N 


16  (cont)  Have  found  procedures  related  to  operation  of _ 

that  were  unnecessarily  difficult,  complicated  or  "unfriendly". 

pi.  the  ACE  to  support  Target  Identification 


ql.  the  ACE  to  support  Target  Refinement 

rl.  the  ACE  to  support  Target  Nomination 

si.  the  ACE  to  support  ISR  requirements 

tl .  the  ACE  to  support  Asset  Management  Responsibilities 


9  =  Unknown 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Human  Factors  Engineering:  Software 


Y/N 


Rating 


17.  Have  found  software  problems  associated  with  operation  of 


al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
forward  tactical  sensors 


fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 
traffic 

hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 
ml .  the  ACE  to  support  All  Source  Fusion 
nl.  the  ACE  to  support  Situation  Development 
o  1 .  the  ACE  to  support  IPB 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 

Function  Focus:  Human  Factors  Engineering:  Software 


Y/N 


17.  Have  found  software  problems  associated  with  operation  of 


pi.  the  ACE  to  support  Target  Identification 

ql.  the  ACE  to  support  Target  Refinement 

rl.  the  ACE  to  support  Target  Nomination 

si.  the  ACE  to  support  ISR  requirements 

tl .  the  ACE  to  support  Asset  Management  Responsibilities 


9  =  Unknown 


Rating 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Y/N 


Function  Focus:  Human  Factors  Engineering:  Placement 


Rating 


18.  Have  found  that  placement  of  equipment,  switches  or  indicators 
in _ made  access  or  use  difficult. 

al.  the  CCS  for  front  end  communications 

bl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
communication  systems  with  the  CCS 

cl.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
communication  systems 

dl.  the  CCS  to  interface  between  Block  II  ACE  All  Source  and 
forward  tactical  sensors 


el.  the  CCS  to  interface  between  Block  II  ACE  Single  Source  and 
forward  tactical  sensors 


fl.  the  CCS  to  receive/relay  of  information  between  (vertical)  and 
within  (horizontal)  echelons 

gl.  the  CCS  to  control  multi-level  security  (MLS)— SCI  and  collateral 
traffic 

hi.  the  ACE  to  support  COMINT 
il.  the  ACE  to  support  ELINT 
j  1 .  the  ACE  to  support  CI/HUMINT 

kl.  the  ACE  to  support  Measurement  and  Signature  Intelligence 
(MASINT) 

11.  the  ACE  to  support  Open  Source  Intelligence  (OSINT) 

ml .  the  ACE  to  support  All  Source  Fusion 

nl.  the  ACE  to  support  Situation  Development 

o  1 .  the  ACE  to  support  IPB 

pi.  the  ACE  to  support  Target  Identification 
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1  =  Completely  Disagree 

2  =  Strongly  Disagree 


3  =  Generally  Disagree 

4  =  Generally  Agree 


5  =  Strongly  Agree 

6  =  Completely  Agree 


9  =  Unknown 


Function  Focus:  Human  Factors  Engineering:  Placement 

Y/N 

18.  Have  found  that  placement  of  equipment,  switches  or  indicators 
in _ made  access  or  use  difficult. 

_  ql .  the  ACE  to  support  Target  Refinement 

_  rl .  the  ACE  to  support  Target  Nomination 

_  si .  the  ACE  to  support  ISR  requirements 

_  tl .  the  ACE  to  support  Asset  Management  Responsibilities 


Rating 
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1  =  Completely  Disagree  3  =  Generally  Disagree  5  =  Strongly  Agree  9  =  Unknown 

2  =  Strongly  Disagree  4  =  Generally  Agree  6  =  Completely  Agree 
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Appendix  I.  Noise  and  Temperature  Measurements  in  and  Around 

the  HETS  M1070  Tractor 


This  appendix  appears  in  its  original  form,  without  editorial  change. 


233 


TABLE  D-15.  ENVIRONMENTAL  DATA  BASED  ON 
AMBIENT  TEMPERATURES 


Reading 

Sample 

Size 

Mean 

Std  dev 

Ambient  temperature  of  5 1 

°  to  60° 

Cab  temperature  (°  F) 

225 

68.36 

7.82 

Cab  noise  (dB) 

225 

78.27 

5.49 

Ambient  temperature 

225 

52.76 

3.49 

Relative  humidity  (pet) 

225 

61.00 

1.72 

Ambient  temperature  of  61° 

to  70°F 

Cab  temperature  (°  F) 

180 

79.91 

6.89 

Cab  noise  (dB) 

180 

79.63 

5.26 

Ambient  temperature 

180 

68.08 

2.26 

Relative  humidity  (pet) 

180 

57.08 

1.39 

Ambient  temperature  of  71° 

to  80°  F 

Cab  temperature  (°  F) 

180 

89.97 

5.69 

Cab  noise  (dB) 

180 

79.49 

4.70 

Ambient  temperature 

180 

76.33 

2.63 

Relative  humidity  (pet) 

180 

56.00 

0.00 

Ambient  temperature  of  81° 

to  90°  F 

Cab  temperature  (°  F) 

180 

94.52 

6.29 

Cab  noise  (dB) 

180 

78.88 

5.43 

Ambient  temperature 

180 

82.33 

1.38 

Relative  humidity  (pet) 

180 

55.67 

1.25 

NOTE:  Army  standards  state  that  hearing  protection  is  recommended 
When  noise  levels  exceed  85  decibels.  Temperature  is  considered 
excessive  when  internal  temperatures  are  10°  higher  than  the 
ambient  temperature. 

TABLE  D-16.  M1070  TRACTOR  (CAB)  TEMPERATURE  AND 
NOISE  FOR  EACH  WINDOW  POSITION 

Temperature  (°  F)  Noise  levels  (db) 

Window  Sample  Sample 


position 

size 

Mean 

Std  dev 

size 

Mean 

Std  dev 

Closed 

255 

85.67 

11.58 

255 

78.55 

5.05 

Half  open 

255 

83.30 

12.17 

255 

78.93 

5.13 

Open 

255 

77.99 

12.18 

255 

79.56 

5.57 
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TABLE  D-17.  M1070  TRACTOR  (CAB)  TEMPERATURE  AND 
NOISE  WHILE  TRAVELING  AT  DIFFERENT  SPEEDS 


Sample 

Reading _ Size  Mean  Std  dev 


0  miles  per  hour  speed 

Cab  temperature  (°  F)  255  79.11  15.23 

Noise  level  (dB)  255  71.51  2.00 

20  miles  per  hour  speed 

Cab  temperature  (°  F)  180  83.04  10.91 

Noise  level  (dB)  180  81.99  1.96 

30  miles  per  hour  speed 

Cab  temperature  (°  F)  180  83.56  10.72 

Noise  level  (dB)  180  82.07  2.10 

40  miles  per  hour  speed 

Cab  temperature  (°  F)  180  84.38  10.56 

Noise  level  (dB)  180  82.36  2.16 


TABLE  D-18.  M1070  TRACTOR  TEMPERATURE  AND  NOISE 
LEVEL  FOR  DIFFERENT  POSITIONS  IN  CAB 

Temperature  (°  F)  Noise  levels  (dB) 

Sample  Sample 


Position 

Size 

Mean 

Std  dev 

Size 

Mean 

Std  dev 

Floor  (near  driver's 
feet) 

153 

82.95 

12.20 

153 

79.35 

5.03 

Driver's  waist  level 

153 

82.19 

12.11 

153 

78.59 

5.29 

Driver's  head  level 
(window  side) 

153 

80.69 

12.06 

153 

78.84 

5.56 

Driver's  head  level 
(doghouse  side) 

153 

84.83 

13.28 

153 

79.28 

4.88 

Floor  near  assistant 
driver's  feet 

153 

80.95 

11.95 

153 

79.00 

5.55 
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Window  Positions 


AMBIENT  TEMPERATURE  (°F) 

Figure  3.  Mean  Predicted  Cab  Temperature  at  Driver’s  Head  Level 
Between  Driver  and  Assistant  Driver  with  Changes  in  Ambient 
Temperature  and  Amount  of  Cab  Window  Closure. 
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Appendix  J.  MANPRINT  Biographical  Questionnaire:  Enhanced  Position 

Location  Reporting  System  (EPLRS) 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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ENHANCED  POSITION  LOCATION  REPORTING  SYSTEM  (EPLRS) 


PL  A  YER/MAINT  AINER 

MANPRINT  BIOGRAPHICAL  QUESTIONNAIRE 

INFORMATION  PRIVACY  ACT  STATEMENT 

A.  Authority:  5  USC  301,  10  USC  3012,  Authority  for  Secretary  of  the  Army  to 
Issue  ARs;  44  USC  3101,  Authority  for  Collecting  Necessary  Data. 

B.  Principal  Purpose:  To  collect  data  to  evaluate  the  effectiveness  of 
soldiers  with  different  backgrounds  to  operate  and  maintain  EPLRS. 

C.  Routine  Uses:  The  data  collected  are  to  be  used  for  research  purposes 
only.  They  will  not  become  a  part  of  any  individual's  record  and  will  not  be 
used  in  whole  or  part  in  making  any  determination  about  an  individual.  Full 
confidentiality  of  responses  will  be  maintained  in  the  processing  of  these 
data. 

D.  Mandatory  or  Voluntary  Disclosure  and  Effect  on  Individual  Not  Providing 
Information:  Voluntary  -  Your  participation  in  this  research  is  strictly 
voluntary.  Individuals  are  encouraged  to  provide  complete  and  accurate 
information  in  the  interests  of  the  research,  but  there  will  be  no  effect  on 
individuals  not  providing  all  or  any  part  of  the  information. 

This  notice  may  be  detached  from  the  rest  of  this  form  and  retained  by 
the  individual  answering  the  questionnaire  if  so  desired. 
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ENHANCED  POSITION  LOCATION  REPORTING  SYSTEM  (EPLRS) 


MANPRINT  BIOGRAPHICAL  PROFILE 


Purpose:  This  form  is  important  because  it  is  about  you!  The  data  you  provide  will  help  Department  of  Defense 
decision-makers  evaluate  how  effectively  EPLRS  has  been  designed  for  soldiers  like  you.  To  do  this,  we  need  you 
to  complete  this  for  accurately.  None  of  your  individual  data  will  be  provided  to  your  chain  of  command  nor 
entered  in  your  personnel  fdes.  Your  responses  will  be  kept  confidential. 

Instructions:  Read  each  item  carefully  and  enter  your  response  in  the  item  space  provided  or  circle  the  appropriate 
response. 


1.  NAME: _ 

(Please  print) 

3.MOS: 

a.  Primary  _ _ _ 

b.  Secondary _ _ 

c.  Duty  _ 

5.  Duty  in  test: 

1  =  Net  Control  Station  (NCS)  Operator 
2=  Net  Control  System(NCS)  Maintainer 
3=  Radio  Set  (RS)  Operator 
4=  Radio  Set  (RS)  Maintainer 
5=  EPLRS  Ground  Reference  Unit  (EGRU) 
Operator 

6=  EPLRS  Test  Set  (ETS)  DS  Maintenance 
Support 

7=  System  Management 

7.  a.  Time  in  Grade _ 

(months) 

b.  Promotable  Status: 

1  =  Yes  2  =  No 


9.  Gender:  1  =  Male  2  =  Female 


11.  Marital  Status: 

1  =  Single 

2  =  Married 

3  =  Separated 

4  =  Divorced 


2.  PIN  NO. _ 

4.  Time  in  primary  MOS: _ 

(months) 


6.  Paygrade:  Please  indicate: 


E- _ 

OR 

O- 


8.  Reenlistment 

a.  Plan  to  reenlist: 

1  =  Yes  2  =  No 

b.  Are  you  sure? 

1  =Yes  2  =  No 

10.  In  my  immediate  family,  I  was  the 

1  =  Only  child 

2  =  Oldest  child 

3  =  Second  or  third  oldest 

4  =  Fourth  or  fifth  oldest 

5  =  More  than  fifth  oldest 
12.  Race: 

1  =  White 

2  =  Black 

3  =  Flispanic 

4  =  American  Indian/ Alaskan 

5  =  Asian/Pacific  Islander 

6  =  Other/unknown 
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13.  Education: 

14.  Place  and  Date  of  Birth: 

1  =  No  high  school  degree 

2  =  High  school  diploma 

a.  City 

3  =  GED 

4  =  Some  college  (1-2  yrs)  /technical  school 

b.  State 

5  =  Two  year  college  degree 

6  =  Three  or  more  years  college,  no  degree 

c.  Country 

7  =  Four  year  college  degree 

d.  Date 

dd  mm  yy 

15.  Size  of  community  I  grew  up  in: 

16.  Handedness  -  write  with: 

1  =  Rural 

1  =  Right 

2  =  Town  (less  than  10,000) 

3  =  Town  (10,000  -  90,000) 

4  =  City  (90,000  -  200,000) 

2  =  Left 

5  =  City  (200,001  -  1  million) 

6  =  More  than  1  million 

3  =  Both  equally  well 

17.  Eye  Dominance: 

18.  Height  (inches) 

1  =  Right  2  =  Left  3  =  Both 

19.  Weight  (pounds): 

20.  Wear  glasses: 

l=Yes  2=  No  3  =  Reading  only 

21.  Hearing: 

22.  Physical  Profile: 

a.  Any  impairment?  1  =  Yes  2  =  No 

a.  Do  you  have  one? 

1  =  Yes  2  =  No 

b.  Corrected?  1  =  Yes  2  =  No 

b.  If  Yes  to  a,  what  is  it  for? 

c.  If  yes  to  b,  how? 

c.  If  Yes  to  a,  is  it: 

1  =  Temporary  2  =  Permanent 

23.  Father's  occupation  while  growing  up: 

24.  Mother's  occupation  while  growing  up: 

25.  a.  Language  used  in  home  while  growing  up? 

1  =  Onlv  English 

2  =  Mainlv  English 

3  =  Occasionally  English 

4  =  Seldom  English 

5  =  Never  English 

b.  Other  language  used? 
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26.  Compared  to  others  in  high  school,  I  thought  of  myself  as: 

1  =  Smarter  than  most  other  students 

2  =  As  smart  as  most  students 

3  =  Not  as  smart  as  most  students 

27.  Before  joining  the  military,  how  often  did  you  read  books  or  magazines  of  your  own  choosing?  (Do  not 
Include  newspapers  or  comics.) 

1  =  Practically  never 

2  =  Seldom 

3  =  Occasionally,  such  as  once  per  week 

4  =  Often,  such  as  at  least  once  a  day 

5  =  Very  often,  such  as  more  than  once  a  day 

28.  In  general.  I'm  the  type  of  person  who  prefers  to  do  a  job: 

1  =  Usually  by  myself 

2  =  Often  by  myself 

3  =  Sometimes  by  myself,  sometimes  with  others 

4  =  Often  with  others 

5  =  Usually  with  others 

29.  Who  succeeds  often  depends  on  luck,  such  as,  who  was  in  the  right  place  at  the  right  time: 

1  =  That's  almost  always  true 

2  =  That's  often  true 

3  =  Sometimes  that's  true 

4  =  That's  seldom  true 

5  =  That's  never  true 

30.  Compared  to  others  in  athletic  ability,  I  am: 

1  =  Less  athletic  than  most  people  in  my  age  group 

2  =  Average  in  athletic  ability 

3  =  Slightly  more  athletic  than  most  people  in  my  age  group 

4  =  In  the  top  15%  of  people  in  my  age  group 

5  =  In  the  top  1%  of  people  in  my  age  group 

31.  Compared  to  other,  I  generally  do  my  job: 

1  =  Slower  than  most  performers 

2  =  Slightly  slower  than  most  performers 

3  =  About  average 

4  =  Slightly  faster  than  most  performers 

5  =  Faster  than  most  performers 

32.  Compared  to  others,  I  generally  do  my  job: 

1  =  More  correctly  than  most  performers 

2  =  Slightly  more  correctly  than  most  performers 

3  =About  average 

4  =  Slightly  less  correctly  than  most  performers 

5  =  Less  correctly  than  most  performers 


241 


33.  To  feel  really  good  mentally,  I  require: 


1=4  hours  sleep  or  less 
2  =  5  to  7  hours  sleep 
3=8  hours  sleep 

4  =  9  hours  sleep 

5  =  10  or  more  hours  sleep 

34.  To  be  successful  at  difficult  jobs  means  sometimes  you  have  to  take  risks. 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 


35.  Nothing  makes  me  more  angry  than  equipment  which  does  NOT  work  reliably. 


1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

36.  It  is  hard  to  exert  job  effort  if  you  work  alone. 


1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

37.  In  important  matters,  I  usually: 


1  =  Rely  on  my  friends  for  information 

2  =  Rely  on  my  family  for  information 

3  =  Rely  on  some  knowledgeable  expert 

4  =  Rely  on  myself  to  find  things  out 


38.  How  well  I  do  my  new  job  significantly  impacts  the  jobs  of  others. 


1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 


39.  Most  people  I  work  with  would  rather  take  more  time  to  do  a  job  right,  than  less  time  to  do  a  "quick  and 
dirty,  it's  good  enough  job" 


1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 
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40.  Most  jobs  are  dull  and  boring. 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

41.  Only  I  am  responsible  for  what  I  will  be. 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

42. 1  know  if  I  persist,  I  can  make  unreliable  equipment  work  properly 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

43.  I'd  rather  be  recognized  as  a  "star"  on  a  losing  team  than  an  unrecognized  member  of  a  winning 
team. 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

44.  People  make  their  own  breaks  in  life. 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

45.  My  greatest  source  of  job  satisfaction  comes  primarily  from: 

1  =  The  people  I  work  with 

2  =  The  high  tech  equipment 

3  =  The  sense  of  accomplishing  something  important 

4  =  Being  successful 

5  =  The  paycheck 

46.  When  traveling  to  a  new  city,  it's  easy  to  get  lost  even  when  you  have  a  map. 

1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 
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47.  People  count  on  me;  I  feel  like  I'm  really  needed  at  work. 


1  =  Strongly  disagree 

2  =  Disagree 
3=1  am  uncertain 

4  =  Agree 

5  =  Strongly  agree 

48.  My  greatest  job  strength  is  my: 

1  =  Physical  endurance 

2  =  Capability  to  learn 

3  =  "Can-do"  attitude 

4  =  Level  of  knowledge 

5  =  Level  of  "know-how"  experience 

49.  If  money  or  family  obligations  were  not  an  obstacle,  I  would  really  like  to  be  a 

1  =  Race  car  driver 

2  =  Helicopter  or  jet  pilot 

3  =  An  accountant 

4  =  Computer  scientist 

5  =  Lawyer 

6  =  Physician 

7  =  Scientist  or  engineer 

8  =  Professor 

9  =  Other _ 

50. 1  intend  to  achieve  the  following  end  of  training  course  score: 

1  =  100% 

2  =  90-99% 

3  =  85-89% 

4  =  80-84% 

5  =  70-79% 

6  =  below  70% 

51.  How  happy  are  you  in  being  selected  to  participate  in  this  test? 

1  =  Very  happy 

2  =  Happy 

3  =  Neither  happy  or  unhappy 

4  =  Unhappy 

5  =  Very  unhappy 


THANK  YOU  FOR  YOUR  COOPERATION 
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